-- Logs begin at Wed 2022-08-24 14:46:42 UTC, end at Wed 2022-08-24 16:05:09 UTC. -- Aug 24 15:12:09.707288 dv-cinder-848900-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 15:12:09.715631 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 15:12:09 2022] *** Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: nodename: dv-cinder-848900-6-iscsi Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: machine: x86_64 Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: clock source: unix Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: pcre jit disabled Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: detected number of CPU cores: 4 Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: current working directory: / Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: detected binary path: /usr/bin/uwsgi-core Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: your processes number limit is 47680 Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: your memory page size is 4096 bytes Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: detected max file descriptor number: 2048 Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: lock engine: pthread robust mutexes Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: thunder lock: enabled Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 15:12:09.722345 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 15:12:09.763685 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: Python main interpreter initialized at 0x5593ef8349c0 Aug 24 15:12:09.763685 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: python threads support enabled Aug 24 15:12:09.763685 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: your server socket listen backlog is limited to 100 connections Aug 24 15:12:09.763685 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: your mercy for graceful operations on workers is 90 seconds Aug 24 15:12:09.764091 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 15:12:09.764091 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: *** Operational MODE: preforking *** Aug 24 15:12:09.764397 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: *** uWSGI is running in multiple interpreter mode *** Aug 24 15:12:09.764397 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: spawned uWSGI master process (pid: 81606) Aug 24 15:12:09.764570 dv-cinder-848900-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 15:12:09.766319 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: spawned uWSGI worker 1 (pid: 81619, cores: 1) Aug 24 15:12:09.766319 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: spawned uWSGI worker 2 (pid: 81620, cores: 1) Aug 24 15:12:09.766319 dv-cinder-848900-6-iscsi devstack@keystone.service[81606]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 15:12:14.047179 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 15:12:14.047179 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: from cryptography.utils import int_from_bytes Aug 24 15:12:14.047973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 15:12:14.047973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: from cryptography.utils import int_from_bytes Aug 24 15:12:14.048415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 15:12:14.048415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: from cryptography.utils import int_from_bytes Aug 24 15:12:14.048534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 15:12:14.048534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: from cryptography.utils import int_from_bytes Aug 24 15:12:14.098171 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 15:12:14.098392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 15:12:14.098554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] command line args: [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 15:12:14.098824 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 15:12:14.098999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ================================================================================ {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 15:12:14.099250 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] admin_token = **** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.099494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] config_dir = [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.099678 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.099763 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 15:12:14.099845 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] config_source = [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.099951 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 15:12:14.100031 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] debug = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.100140 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] command line args: [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 15:12:14.100226 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.100450 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 15:12:14.100570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.100660 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ================================================================================ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 15:12:14.100722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.100784 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] admin_token = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.100981 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.101095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] config_dir = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.101196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.101310 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.101408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] config_source = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.101500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] list_limit = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.101592 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] debug = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.101685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] log_config_append = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.101793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.101913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.102014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] log_dir = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.102121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.102245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] log_file = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.102360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.102477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.102589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.102714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.102795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.102919 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.103000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.103000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_config_append = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.103144 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.103217 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.103339 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.103415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_dir = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.103530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.103615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_file = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.103738 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.103813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.103926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.104000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.104121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.104208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.104287 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.104413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.104501 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.104584 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.104733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.104807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.104973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.105051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.105215 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.105342 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.105463 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.105585 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.105707 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.105883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.106031 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.76/identity {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.106161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.106308 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] publish_errors = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.106425 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.106539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.106653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.106773 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.106893 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.107008 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.107121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.107239 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.107352 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.76/identity {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.107474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] standard_threads = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.107595 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] publish_errors = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.107710 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.107840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.107967 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.108088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.108208 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.108329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.108455 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] use_journal = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.108569 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.108682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] use_json = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.108682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] use_stderr = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.108902 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.109016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] use_syslog = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.109180 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] standard_threads = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.109302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.109417 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.109530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.109642 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.109763 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.109886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.109886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_journal = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.110099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.110233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_json = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.110360 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.110476 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_stderr = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.110476 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] use_syslog = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.110702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.110816 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 15:12:14.110929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.111041 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.111160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.111282 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.111402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.111516 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.111629 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.111749 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.111871 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.111994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.112115 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.112229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.112342 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.112454 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.112567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.112696 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.112810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.112930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.113044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.113156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.113268 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.113390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.113508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.113622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.113735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.113854 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.113976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.114099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.114316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.114435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.114559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.114681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.114803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.114995 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.115113 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.115226 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.115340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.115452 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.115574 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.115689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.115802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.115915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.116035 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.116156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.116279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.116394 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.116507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.116620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.116733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.116846 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.116971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.117093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.117207 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.117303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.117420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.117534 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.117655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.117776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.117897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.118017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.118131 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.118259 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.118387 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.118503 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.118623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.118737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.118858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.118979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.119105 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.119220 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.119334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.119446 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.119559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.119672 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.119797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.119912 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.120025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.120143 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.120259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.120372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.120560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.120676 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.120789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.120902 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.121015 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.121127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.121127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.121357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.121477 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.121597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.121711 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.121831 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.121953 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.122068 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.122181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.122309 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.122434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.122552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.122676 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.122780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.122877 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.122975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.123070 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.123163 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.123266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.123362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.123457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.123592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.123592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.123592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.123875 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.123970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.124073 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.124192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.124308 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.124308 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.124531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.124641 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.124760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.124760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.124975 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.125087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.125207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.125334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.125450 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.125450 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.125672 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.125788 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.125903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.126023 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.126146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.126314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.126439 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.126556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.126670 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.126792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.126907 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.127021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.127124 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.127237 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.127356 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.127356 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.127582 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.127704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.127819 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.127932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.128046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.128167 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.128281 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.128401 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.128519 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.128633 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.128747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.128868 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.128983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.129103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.129218 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.129331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.129445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.129567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.129690 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.129805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.129919 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.129919 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.130132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.130272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.130390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.130505 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.130619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.130619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.130835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.130957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.131073 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.131187 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.131307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.131421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.131535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.131726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.131851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.131967 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.132088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.132204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.132327 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.132452 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.132582 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.133201 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.133317 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.133317 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.133532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.133652 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.133777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.133893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.134007 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.134121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.134261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.134380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.134507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.134624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.134738 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] auth.external = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.134853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.134968 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.135088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.135212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.135329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.135443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.135558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.135673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] auth.password = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.135787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.135935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] auth.token = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.136061 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.external = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.136190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.136315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.136432 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.136547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.136663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.136789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.136914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.137031 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.password = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.137147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.137262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] auth.token = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.137378 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.137493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.137616 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.137733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.137848 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] credential.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.137964 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.138080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.138195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.138337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.138464 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.138583 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.138699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.138815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.138930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.139054 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.139162 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.139278 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.139400 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.139517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.139632 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.139759 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.139876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.139992 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.140108 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.140225 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.140341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.140465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.140587 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.140704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.140820 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.140934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.141050 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.141174 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.141293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.141409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.141409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] federation.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.141627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.141743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.141866 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.141983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.142098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.142213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.142390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.142508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.142636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.142754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.142870 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.142985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.143100 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.143215 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.143340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.143456 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.143572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.143686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.143802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.143925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.144049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.144166 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.144281 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.144398 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.144513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.144635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.144752 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.144868 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.144983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.145104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.145222 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.145346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.145464 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.145580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.145695 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.145819 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.145935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.146127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.146260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.146379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.146508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.146626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.146742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.146866 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.146984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.147099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.147215 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.147329 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.147444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.147569 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.147693 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.147810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.147926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.148041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.148156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.148314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.148954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.149058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.149156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.149156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.149356 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.149356 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.149832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.149952 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.150067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.150182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.150356 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.150486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.150604 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.150728 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.150846 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.150961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.151076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.151200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.151317 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.151433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.151548 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.151663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.151779 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.151903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.152020 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.152135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.152251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.152366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.152485 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.152613 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.152731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.152847 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.152962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.153067 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.153183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.153310 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.153428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.153550 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.153668 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.153783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.153899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.154024 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.154141 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.154272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.154389 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.154505 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.154621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.154621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.154854 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.154854 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.155071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.155071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.155295 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.155412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.155527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.155642 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.155756 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.155871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.155995 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.156112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.156227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.156341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.156456 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.156572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.156695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.156813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.156928 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.157044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.157159 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.157274 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.157397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.157520 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.157637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.157752 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.157868 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.157984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.158107 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.158253 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.158372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.158487 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.158602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.158717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.158850 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.158968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.159085 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.159085 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.159310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.159425 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.159550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.159667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.159789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.159906 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.160022 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.160138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.160261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.160378 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.160493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.160609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.160724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.160839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.160963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.161080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.161196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.161311 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.161425 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.161540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.161664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.161780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.161896 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.162011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.162126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.162260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.162391 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.162514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.162630 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.162745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.162860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.162975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.163099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.163215 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.163331 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.163446 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.163560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.163675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.163798 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.163915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.164030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.164145 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.164259 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.164375 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.164499 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.164615 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.164730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.164844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.164959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.165081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.165205 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.165322 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.165438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.165552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.165666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.165781 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.165904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.166027 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] resource.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.166144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.166296 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.166415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.166530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.166657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.166775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.166890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.167005 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.167120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.167235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.167359 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.167476 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.167592 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.167713 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.167830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.167945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.168069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.168186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.168301 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.168415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] role.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.168531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.168645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] role.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.168769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.168885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.169000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.169115 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.169229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.169418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.169539 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.169654 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.169769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.169884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.170000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] role.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.170123 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.170254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] role.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.170372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.170487 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.170602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.170717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.170844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.170961 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.171077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.171192 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.171307 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.171422 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.171546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.171664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.171780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.171895 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.172009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.172125 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.172125 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.172354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.172471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.172586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.172702 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.172818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.172941 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.173058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.173173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.173289 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.173404 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.173519 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.173519 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.173748 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.173847 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.173944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.174041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.174138 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.174302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.174426 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.174543 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.174658 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.174775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.174899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.175015 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.175131 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.175252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.175369 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.175484 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.175484 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.175711 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.175828 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.175943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.176058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.176173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.176173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.176399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.176514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.176629 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] token.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.176629 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.176845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.176969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.177085 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.177200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.177315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.177315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.177531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.177654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.177772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.177887 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.178002 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.178117 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.178312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.178444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.178562 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.178677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.178792 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.178907 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.179022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.179146 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.179263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.179378 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.179493 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.179608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.179722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.179845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.179968 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.180085 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.180201 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.180201 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.180417 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.180541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.180658 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.180773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.connection = **** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.180898 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.181023 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.181141 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.181256 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.181379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.181379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.181597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.181713 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.181828 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.181943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.182067 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.182185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.182332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.182454 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.182582 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.182709 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.182826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.182949 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.183074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.183204 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.183322 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.183438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.183553 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.183669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.183792 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.183910 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.184025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.184140 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.184256 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.184371 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.184497 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.184614 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.184730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.184845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.184960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.185075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.185199 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.185199 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81619) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 15:12:14.185419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.185419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.185419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.185419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.185419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.185419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.185419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 15:12:14.185419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81620) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 15:12:14.214499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.214725 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.214949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.218907 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.219137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.219358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.223489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.228093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.251776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.252006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.254146 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.254395 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.305985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.307096 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.354464 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.354701 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.355970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.356207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.511070 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.511253 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.511336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.511400 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.511486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.511631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.511723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.511845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.511929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.512067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.512160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.512286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.535827 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.535975 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.536062 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.536126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.536274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.536357 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.536498 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.536581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.536720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.536812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.536933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.537024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.537147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.537231 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.548893 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.549164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.561751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.561929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.562124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.562209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.562385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.562484 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.562602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.562683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.562817 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.562900 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.563032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81619) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.563123 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.563244 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.563460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81620) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 15:12:14.579614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.579824 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 15:12:14.580169 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.580368 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 15:12:14.580473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.580640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.580722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 15:12:14.580903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.581048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.581201 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 15:12:14.581413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.581494 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.581680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.581769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.582248 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.582528 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.591585 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.591868 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.592139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.592422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.592504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.592734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.593009 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.593296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.599426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.600394 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.602893 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.603179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.603465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.603748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.603916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.604032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.604204 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.604285 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.604487 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.604577 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.604771 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.604853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.605055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.605179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.605318 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.605457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.605590 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.605731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.605865 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.606274 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.606557 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.606829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.628036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.629283 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.629497 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.629782 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.630789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.631078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.635001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.635284 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.635565 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.636299 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.636592 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.636881 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.641830 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.642118 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.643235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.643526 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.646197 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.646511 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.646797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.647084 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.647366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.647620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.647712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.647900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.647990 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.648171 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.648252 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.649043 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.649326 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.649861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.650133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.650425 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.666174 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.666480 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.667625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.667911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.670331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.671799 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.673539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.673835 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.675018 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.675321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.678005 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.679592 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.681235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.682745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.684498 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.684797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.685081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.685363 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.685639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.685931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.686013 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.686338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.686586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.686878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.687163 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.687456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.702234 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.702541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.702812 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.703078 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.703755 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.704056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.704328 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.704594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.709929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.711520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.711606 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.711801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.713083 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.713366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.714472 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.714761 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.715053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.715347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.715625 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.716027 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.716322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.716617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.716904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.717203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.727429 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.727714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.727995 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.728279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.728561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.728841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.728982 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.729118 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.729272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.729559 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.729846 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.730129 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.730456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.730747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.744384 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.746057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.747423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.749466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.750994 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.751772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.752016 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.753248 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.753472 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.753557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.753842 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.754971 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.755263 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.755557 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.760830 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.762313 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.762598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.764066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.765336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.765636 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.765927 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.766237 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.766965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.767262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.767564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.767850 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.774917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.775208 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.775491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.776610 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.776905 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.777190 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.782332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81619) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.782619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.782904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.783183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.783470 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.783754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.784032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.784114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81620) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 15:12:14.784306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.784391 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.784585 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.784666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.784863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.784946 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.785135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.785217 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.785409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.785499 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.785686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.785767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.785958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81619) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.786136 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.786244 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.786541 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.786818 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.787090 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.787359 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.787620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81620) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 15:12:14.819756 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5593ef8349c0 pid: 81619 (default app) Aug 24 15:12:14.821970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x5593ef8349c0 pid: 81620 (default app) Aug 24 15:12:14.823709 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441d537-f7c5-449a-8ecc-73ddb309d0eb None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:14.823994 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441d537-f7c5-449a-8ecc-73ddb309d0eb None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:14.824267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5441d537-f7c5-449a-8ecc-73ddb309d0eb None None] PATH_INFO: `/v3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:14.828457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1/1] 10.222.0.76 () {50 vars in 791 bytes} [Wed Aug 24 15:12:14 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:21.383254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3c953-b840-4188-9fc1-cb0818dda522 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:21.383771 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3c953-b840-4188-9fc1-cb0818dda522 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:21.383771 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3c953-b840-4188-9fc1-cb0818dda522 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:21.388068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1/2] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:21.394549 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcf086-b137-4e02-b360-357c6b8af29d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:21.395021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcf086-b137-4e02-b360-357c6b8af29d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:21.395160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcf086-b137-4e02-b360-357c6b8af29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:21.465628 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: INFO dbcounter [None req-bedcf086-b137-4e02-b360-357c6b8af29d None None] Registered counter for database keystone Aug 24 15:12:21.484554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG oslo_db.sqlalchemy.engines [None req-bedcf086-b137-4e02-b360-357c6b8af29d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81619) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 15:12:21.490772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writer thread running {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 15:12:21.670934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG passlib.handlers.bcrypt [None req-bedcf086-b137-4e02-b360-357c6b8af29d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81619) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 15:12:21.671348 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG passlib.handlers.bcrypt [None req-bedcf086-b137-4e02-b360-357c6b8af29d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81619) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 15:12:21.725766 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-bedcf086-b137-4e02-b360-357c6b8af29d None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:21.727422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bedcf086-b137-4e02-b360-357c6b8af29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:21.859334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 2/3] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:21 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 15:12:22.333755 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3c953-b840-4188-9fc1-cb0818dda522 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:22.402318 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: INFO dbcounter [None req-faf3c953-b840-4188-9fc1-cb0818dda522 None None] Registered counter for database keystone Aug 24 15:12:22.423958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG oslo_db.sqlalchemy.engines [None req-faf3c953-b840-4188-9fc1-cb0818dda522 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81620) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 15:12:22.430639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writer thread running {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 15:12:22.549028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf3c953-b840-4188-9fc1-cb0818dda522 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:22.553342 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06adb2bc-6010-4299-a904-f093b83a5147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:22.553970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adb2bc-6010-4299-a904-f093b83a5147 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:22.554268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adb2bc-6010-4299-a904-f093b83a5147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:22.554565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06adb2bc-6010-4299-a904-f093b83a5147 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:22.562669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06adb2bc-6010-4299-a904-f093b83a5147 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:22.614287 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: WARNING keystone.common.rbac_enforcer.enforcer [None req-06adb2bc-6010-4299-a904-f093b83a5147 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 15:12:22.614573 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06adb2bc-6010-4299-a904-f093b83a5147 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:22.624242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 2/4] 10.222.0.76 () {58 vars in 1324 bytes} [Wed Aug 24 15:12:22 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 15:12:22.630248 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcf086-b137-4e02-b360-357c6b8af29d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:22.644545 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcf086-b137-4e02-b360-357c6b8af29d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:22.650005 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f93f46-95fc-457b-96e6-938eda986768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:22.651130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f93f46-95fc-457b-96e6-938eda986768 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:22.651408 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f93f46-95fc-457b-96e6-938eda986768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:22.651677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f93f46-95fc-457b-96e6-938eda986768 admin admin] PATH_INFO: `/v3/projects/e70c51040e434074a3d61be0b7c8533b` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:22.653201 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f93f46-95fc-457b-96e6-938eda986768 admin admin] RBAC: Authorizing `identity:get_project(project_id=e70c51040e434074a3d61be0b7c8533b)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:22.706420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: WARNING keystone.common.rbac_enforcer.enforcer [None req-79f93f46-95fc-457b-96e6-938eda986768 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 15:12:22.706772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f93f46-95fc-457b-96e6-938eda986768 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:22.711790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 3/5] 10.222.0.76 () {56 vars in 1207 bytes} [Wed Aug 24 15:12:22 2022] GET /identity/v3/projects/e70c51040e434074a3d61be0b7c8533b => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:12:24.599606 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980c99e-70ca-4a80-ab92-51df1ece14ed None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:24.600142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980c99e-70ca-4a80-ab92-51df1ece14ed None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:24.600142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980c99e-70ca-4a80-ab92-51df1ece14ed None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:24.604746 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 3/6] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:24.610251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c41db3-e8ed-4356-ab52-17c30b60041e None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:24.610692 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c41db3-e8ed-4356-ab52-17c30b60041e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:24.610822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c41db3-e8ed-4356-ab52-17c30b60041e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:24.631036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-80c41db3-e8ed-4356-ab52-17c30b60041e None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:24.632597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-80c41db3-e8ed-4356-ab52-17c30b60041e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:24.640454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 4/7] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:24 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 15:12:24.913747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980c99e-70ca-4a80-ab92-51df1ece14ed None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:24.926135 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980c99e-70ca-4a80-ab92-51df1ece14ed None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:24.931087 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2b595-89de-4f61-881a-a6c2ba7f6133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:24.931687 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2b595-89de-4f61-881a-a6c2ba7f6133 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:24.931966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2b595-89de-4f61-881a-a6c2ba7f6133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:24.932237 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2b595-89de-4f61-881a-a6c2ba7f6133 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:24.939876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2b595-89de-4f61-881a-a6c2ba7f6133 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:24.941663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2b595-89de-4f61-881a-a6c2ba7f6133 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:24.951078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 4/8] 10.222.0.76 () {58 vars in 1324 bytes} [Wed Aug 24 15:12:24 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:12:24.956599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c41db3-e8ed-4356-ab52-17c30b60041e None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:24.966371 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c41db3-e8ed-4356-ab52-17c30b60041e None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:24.970730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ef191c-614e-4740-81c5-31949d667592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:24.971354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef191c-614e-4740-81c5-31949d667592 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:24.971634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef191c-614e-4740-81c5-31949d667592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:24.971910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ef191c-614e-4740-81c5-31949d667592 admin admin] PATH_INFO: `/v3/users/793b87d316d544f9aecda76f798927dc` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:24.973233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef191c-614e-4740-81c5-31949d667592 admin admin] RBAC: Authorizing `identity:get_user(user_id=793b87d316d544f9aecda76f798927dc)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:24.975087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ef191c-614e-4740-81c5-31949d667592 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:24.976715 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 5/9] 10.222.0.76 () {56 vars in 1198 bytes} [Wed Aug 24 15:12:24 2022] GET /identity/v3/users/793b87d316d544f9aecda76f798927dc => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:26.947187 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffa896-bf22-4509-a0ac-b5f5257beb1e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:26.947727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffa896-bf22-4509-a0ac-b5f5257beb1e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:26.947727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ffa896-bf22-4509-a0ac-b5f5257beb1e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:26.949074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 5/10] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:12:26.954385 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9492a-ed29-4278-82b2-eabd31a08d60 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:26.954797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9492a-ed29-4278-82b2-eabd31a08d60 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:26.955050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9492a-ed29-4278-82b2-eabd31a08d60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:26.962708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e977c62-b8a2-4929-a1b8-d7feefb10f24 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:26.962994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e977c62-b8a2-4929-a1b8-d7feefb10f24 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:26.963266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e977c62-b8a2-4929-a1b8-d7feefb10f24 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:26.964549 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 6/11] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:12:26.969742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aaa0e-06e0-45ef-bcb6-55e3f1074e7b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:26.970095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aaa0e-06e0-45ef-bcb6-55e3f1074e7b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:26.970409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17aaa0e-06e0-45ef-bcb6-55e3f1074e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:26.973969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-80a9492a-ed29-4278-82b2-eabd31a08d60 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:26.975196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-80a9492a-ed29-4278-82b2-eabd31a08d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:27.012196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG passlib.handlers.bcrypt [None req-a17aaa0e-06e0-45ef-bcb6-55e3f1074e7b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81620) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 15:12:27.012620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG passlib.handlers.bcrypt [None req-a17aaa0e-06e0-45ef-bcb6-55e3f1074e7b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81620) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 15:12:27.055272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 6/12] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:27.058729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a17aaa0e-06e0-45ef-bcb6-55e3f1074e7b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:27.060357 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a17aaa0e-06e0-45ef-bcb6-55e3f1074e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:27.157893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 7/13] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:12:27.322741 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9492a-ed29-4278-82b2-eabd31a08d60 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:27.347997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:27.424014 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17aaa0e-06e0-45ef-bcb6-55e3f1074e7b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:27.438176 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9492a-ed29-4278-82b2-eabd31a08d60 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:27.474153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904550a-3a19-478a-a6c1-11184151b237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:27.474837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904550a-3a19-478a-a6c1-11184151b237 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:27.475124 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904550a-3a19-478a-a6c1-11184151b237 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:27.475403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6904550a-3a19-478a-a6c1-11184151b237 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:27.488292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6904550a-3a19-478a-a6c1-11184151b237 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:27.497961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-6904550a-3a19-478a-a6c1-11184151b237 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:12:27.497961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:27.497961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:27.497961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:27.497961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:27.497961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:27.497961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:27.497961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:27.497961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:27.497961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:27.497961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:27.497961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:27.497961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:27.497961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:27.497961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:27.497961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:27.499114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:27.499114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:27.499114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:27.499114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:27.499114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:27.499114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:27.499114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:27.499114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:27.499114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:27.499114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:27.499114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:27.499114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:27.499114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:27.499114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:27.499114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:27.499114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:27.500081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:27.500081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:27.500081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:27.500081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:27.500081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:27.500081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:27.500081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:27.500081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:27.500081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:27.500081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:27.500081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:12:27.500081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:12:27.500081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 7/14] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:12:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:27.505905 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904550a-3a19-478a-a6c1-11184151b237 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:27.536677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17aaa0e-06e0-45ef-bcb6-55e3f1074e7b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:27.540096 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:27.577982 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266d282-aeb8-45b2-919c-588c95693b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:27.578802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266d282-aeb8-45b2-919c-588c95693b72 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:27.579087 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266d282-aeb8-45b2-919c-588c95693b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:27.579355 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1266d282-aeb8-45b2-919c-588c95693b72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:27.593403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:27.612377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6904550a-3a19-478a-a6c1-11184151b237 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:27.646369 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1266d282-aeb8-45b2-919c-588c95693b72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:27.646720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959d188-944e-45c2-88a8-fc6380d08b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:27.647332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959d188-944e-45c2-88a8-fc6380d08b81 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:27.647610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959d188-944e-45c2-88a8-fc6380d08b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:27.647884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959d188-944e-45c2-88a8-fc6380d08b81 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:27.648008 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1266d282-aeb8-45b2-919c-588c95693b72 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:27.648571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959d188-944e-45c2-88a8-fc6380d08b81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:27.648871 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959d188-944e-45c2-88a8-fc6380d08b81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:27.650576 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959d188-944e-45c2-88a8-fc6380d08b81 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:27.662265 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 8/15] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:12:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:12:27.668435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959d188-944e-45c2-88a8-fc6380d08b81 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:27.746572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 8/16] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:12:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:12:27.752157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266d282-aeb8-45b2-919c-588c95693b72 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:27.755662 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:27.769376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959d188-944e-45c2-88a8-fc6380d08b81 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:27.804720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaa290-7330-4c22-90d2-c27f07f8e58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:27.805389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaa290-7330-4c22-90d2-c27f07f8e58b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:27.805700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaa290-7330-4c22-90d2-c27f07f8e58b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:27.805973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27eaa290-7330-4c22-90d2-c27f07f8e58b None admin] PATH_INFO: `/v3/users/793b87d316d544f9aecda76f798927dc` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:27.807286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eaa290-7330-4c22-90d2-c27f07f8e58b None admin] RBAC: Authorizing `identity:get_user(user_id=793b87d316d544f9aecda76f798927dc)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:27.808836 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27eaa290-7330-4c22-90d2-c27f07f8e58b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:27.810670 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 9/17] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:27 2022] GET /identity/v3/users/793b87d316d544f9aecda76f798927dc => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:12:27.816387 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaa290-7330-4c22-90d2-c27f07f8e58b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:27.832147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:27.862863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1266d282-aeb8-45b2-919c-588c95693b72 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:27.901360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d934f23-2aac-4ca9-9e3b-31e49e208840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:27.902253 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d934f23-2aac-4ca9-9e3b-31e49e208840 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:27.902545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d934f23-2aac-4ca9-9e3b-31e49e208840 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:27.902887 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d934f23-2aac-4ca9-9e3b-31e49e208840 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:27.923348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d934f23-2aac-4ca9-9e3b-31e49e208840 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:27.924840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d934f23-2aac-4ca9-9e3b-31e49e208840 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:27.930311 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27eaa290-7330-4c22-90d2-c27f07f8e58b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:27.946386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7d934f23-2aac-4ca9-9e3b-31e49e208840 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:12:27.946386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:27.946386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:27.946386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:27.946386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:27.946386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:27.946386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:27.946386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:27.946386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:27.946386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:27.946386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:27.946386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:27.946386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:12:27.946386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:12:27.946386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:12:27.946386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:27.947959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:12:27.949074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:12:27.949074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:12:27.949074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:12:27.949074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 9/18] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:12:27.953395 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d934f23-2aac-4ca9-9e3b-31e49e208840 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:27.967720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39512a03-231a-413e-8111-2e0cd3699002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:27.968356 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39512a03-231a-413e-8111-2e0cd3699002 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:27.968638 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39512a03-231a-413e-8111-2e0cd3699002 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:27.969634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39512a03-231a-413e-8111-2e0cd3699002 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:27.970021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39512a03-231a-413e-8111-2e0cd3699002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:27.971539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39512a03-231a-413e-8111-2e0cd3699002 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:27.973099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 10/19] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:12:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:12:27.979115 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39512a03-231a-413e-8111-2e0cd3699002 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:28.017479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:28.064116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d934f23-2aac-4ca9-9e3b-31e49e208840 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:28.066740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:28.086140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39512a03-231a-413e-8111-2e0cd3699002 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:28.098284 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942df3e5-7914-4d01-b217-831c4aaa1861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:28.098901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942df3e5-7914-4d01-b217-831c4aaa1861 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:28.099179 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942df3e5-7914-4d01-b217-831c4aaa1861 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:28.099490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942df3e5-7914-4d01-b217-831c4aaa1861 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:28.100125 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942df3e5-7914-4d01-b217-831c4aaa1861 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:28.100428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942df3e5-7914-4d01-b217-831c4aaa1861 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:28.102365 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942df3e5-7914-4d01-b217-831c4aaa1861 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:28.121117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db4e672-73a1-43d3-9cfe-c4a50179564c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:28.121312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 10/20] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:12:28.122026 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4e672-73a1-43d3-9cfe-c4a50179564c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:28.122338 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4e672-73a1-43d3-9cfe-c4a50179564c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:28.122612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db4e672-73a1-43d3-9cfe-c4a50179564c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:28.123340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4e672-73a1-43d3-9cfe-c4a50179564c None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.domain.id=default, user.id=793b87d316d544f9aecda76f798927dc) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:28.123705 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4e672-73a1-43d3-9cfe-c4a50179564c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:28.125188 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db4e672-73a1-43d3-9cfe-c4a50179564c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:28.132815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 11/21] 10.222.0.76 () {56 vars in 1322 bytes} [Wed Aug 24 15:12:27 2022] GET /identity/v3/role_assignments?user.id=793b87d316d544f9aecda76f798927dc&scope.domain.id=default&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 233 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:12:30.423837 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd330533-f629-4389-be4d-ade5b1cf1feb None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:30.424723 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd330533-f629-4389-be4d-ade5b1cf1feb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:30.425156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd330533-f629-4389-be4d-ade5b1cf1feb None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:30.426674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 11/22] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:30.439712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0743e0-27bd-4a2c-9c7c-abf4f2b4a99f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:30.440486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0743e0-27bd-4a2c-9c7c-abf4f2b4a99f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:30.440922 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0743e0-27bd-4a2c-9c7c-abf4f2b4a99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:30.466560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6c0743e0-27bd-4a2c-9c7c-abf4f2b4a99f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:30.468019 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6c0743e0-27bd-4a2c-9c7c-abf4f2b4a99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:30.489325 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a544990-78ce-4994-a343-edf921d32750 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:30.489928 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a544990-78ce-4994-a343-edf921d32750 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:30.490436 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a544990-78ce-4994-a343-edf921d32750 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:30.492047 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 12/23] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:12:30.503354 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa572e0-31d7-4092-898a-269daa8cbb74 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:30.504051 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa572e0-31d7-4092-898a-269daa8cbb74 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:30.504518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa572e0-31d7-4092-898a-269daa8cbb74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:30.529967 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-bfa572e0-31d7-4092-898a-269daa8cbb74 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:30.533145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bfa572e0-31d7-4092-898a-269daa8cbb74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:30.607534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 12/24] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:30.634610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db061e4-d386-4b33-aeaf-e39f77c838b6 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:30.635110 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db061e4-d386-4b33-aeaf-e39f77c838b6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:30.635244 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 13/25] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:12:30.635690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db061e4-d386-4b33-aeaf-e39f77c838b6 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:30.637214 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 13/26] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:12:30.651996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c02117-fc68-4948-ae8f-7c2f132906af None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:30.652283 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c02117-fc68-4948-ae8f-7c2f132906af None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:30.652557 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c02117-fc68-4948-ae8f-7c2f132906af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:30.678274 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a9c02117-fc68-4948-ae8f-7c2f132906af None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:30.679951 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a9c02117-fc68-4948-ae8f-7c2f132906af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:30.734239 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:30.764891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 14/27] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:30.781343 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6bd3f-15ca-4171-b00e-b6759f5a9a6d None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:30.781919 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6bd3f-15ca-4171-b00e-b6759f5a9a6d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:30.782425 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6bd3f-15ca-4171-b00e-b6759f5a9a6d None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:30.784887 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 14/28] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:30.789039 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135aaaa-13a6-4c48-9eb2-7d3bdf6bf3c1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:30.789315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135aaaa-13a6-4c48-9eb2-7d3bdf6bf3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:30.789588 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f135aaaa-13a6-4c48-9eb2-7d3bdf6bf3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:30.818155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f135aaaa-13a6-4c48-9eb2-7d3bdf6bf3c1 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:30.819381 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f135aaaa-13a6-4c48-9eb2-7d3bdf6bf3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:30.912761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 15/29] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:31.155328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6bd3f-15ca-4171-b00e-b6759f5a9a6d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:31.233453 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135aaaa-13a6-4c48-9eb2-7d3bdf6bf3c1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:31.246527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:31.282061 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6bd3f-15ca-4171-b00e-b6759f5a9a6d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:31.323305 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd63de-c9b5-44c0-b894-fd4edb1ec814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:31.324253 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd63de-c9b5-44c0-b894-fd4edb1ec814 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:31.324765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd63de-c9b5-44c0-b894-fd4edb1ec814 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:31.325309 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cd63de-c9b5-44c0-b894-fd4edb1ec814 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:31.326181 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd63de-c9b5-44c0-b894-fd4edb1ec814 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:31.327997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cd63de-c9b5-44c0-b894-fd4edb1ec814 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:31.430533 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f135aaaa-13a6-4c48-9eb2-7d3bdf6bf3c1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:31.437891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:31.468657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 15/30] 10.222.0.76 () {60 vars in 1131 bytes} [Wed Aug 24 15:12:31 2022] POST /identity/v3/roles => generated 225 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:12:31.470502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c06e8f-a57f-4eb2-bee6-1b460041a72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:31.470910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd63de-c9b5-44c0-b894-fd4edb1ec814 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:31.471173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c06e8f-a57f-4eb2-bee6-1b460041a72e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:31.471405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c06e8f-a57f-4eb2-bee6-1b460041a72e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:31.471677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c06e8f-a57f-4eb2-bee6-1b460041a72e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:31.483125 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c06e8f-a57f-4eb2-bee6-1b460041a72e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:31.485227 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c06e8f-a57f-4eb2-bee6-1b460041a72e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:31.514330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-22c06e8f-a57f-4eb2-bee6-1b460041a72e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:12:31.514330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:31.514330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:31.514330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:31.514330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:31.514330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:31.514330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:31.514330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:31.514330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:31.514330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:31.514330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:31.514330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:31.514330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:12:31.514330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:12:31.514330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:12:31.514330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:31.515434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:12:31.516408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:12:31.516408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:12:31.516408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:12:31.516408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 16/31] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:12:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:12:31.516672 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c06e8f-a57f-4eb2-bee6-1b460041a72e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:31.615720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cd63de-c9b5-44c0-b894-fd4edb1ec814 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:31.653767 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed785bd5-4a23-4e00-9188-5a9d0d1a8e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:31.658242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed785bd5-4a23-4e00-9188-5a9d0d1a8e27 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:31.658242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed785bd5-4a23-4e00-9188-5a9d0d1a8e27 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:31.658242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed785bd5-4a23-4e00-9188-5a9d0d1a8e27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:31.666680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed785bd5-4a23-4e00-9188-5a9d0d1a8e27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:31.672592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ed785bd5-4a23-4e00-9188-5a9d0d1a8e27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:12:31.672592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:31.672592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:31.672592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:31.672592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:31.672592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:31.672592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:31.672592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:31.672592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:31.672592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:31.672592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:31.672592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:31.672592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:31.672592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:31.672592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:31.672592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:31.673431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:31.673431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:31.673431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:31.673431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:31.673431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:31.673431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:31.673431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:31.673431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:31.673431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:31.673431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:31.673431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:31.673431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:31.673431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:31.673431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:31.673431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:31.673431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:31.674258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:31.674258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:31.674258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:31.674258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:31.674258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:31.674258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:31.674258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:31.674258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:31.674258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:31.674258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:31.674258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:12:31.674258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:12:31.674258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 16/32] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:12:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:31.674884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed785bd5-4a23-4e00-9188-5a9d0d1a8e27 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:31.680169 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:31.700796 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c06e8f-a57f-4eb2-bee6-1b460041a72e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:31.765666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa00f7-abef-4a03-869d-b8265919b1a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:31.766332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa00f7-abef-4a03-869d-b8265919b1a3 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:31.766613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa00f7-abef-4a03-869d-b8265919b1a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:31.766881 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa00f7-abef-4a03-869d-b8265919b1a3 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:31.767547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa00f7-abef-4a03-869d-b8265919b1a3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:31.768922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa00f7-abef-4a03-869d-b8265919b1a3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:31.805903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed785bd5-4a23-4e00-9188-5a9d0d1a8e27 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:31.843709 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde8e46-fcc7-4237-aa78-52ae677aec5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:31.844420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde8e46-fcc7-4237-aa78-52ae677aec5f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:31.844707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde8e46-fcc7-4237-aa78-52ae677aec5f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:31.844981 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde8e46-fcc7-4237-aa78-52ae677aec5f None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:31.845691 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde8e46-fcc7-4237-aa78-52ae677aec5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:31.845999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde8e46-fcc7-4237-aa78-52ae677aec5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:31.847723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcde8e46-fcc7-4237-aa78-52ae677aec5f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:31.868167 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 17/33] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:12:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:31.869227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde8e46-fcc7-4237-aa78-52ae677aec5f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:31.901721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 17/34] 10.222.0.76 () {60 vars in 1131 bytes} [Wed Aug 24 15:12:31 2022] POST /identity/v3/roles => generated 219 bytes in 386 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:12:31.903382 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa00f7-abef-4a03-869d-b8265919b1a3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:31.926013 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:31.954696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:32.041509 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde8e46-fcc7-4237-aa78-52ae677aec5f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:32.081065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5a7a3-b4b4-43cb-997d-095148456839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:32.081686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5a7a3-b4b4-43cb-997d-095148456839 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:32.081967 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5a7a3-b4b4-43cb-997d-095148456839 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:32.082260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5a7a3-b4b4-43cb-997d-095148456839 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:32.082945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5a7a3-b4b4-43cb-997d-095148456839 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:32.083247 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5a7a3-b4b4-43cb-997d-095148456839 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:32.085240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5a7a3-b4b4-43cb-997d-095148456839 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:32.096057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 18/35] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:12:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:32.097453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93855985-a598-472a-8085-f7044cf0e150 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:32.097748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93855985-a598-472a-8085-f7044cf0e150 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:32.098023 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93855985-a598-472a-8085-f7044cf0e150 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:32.099419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 19/36] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:32.102978 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa00f7-abef-4a03-869d-b8265919b1a3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:32.105713 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c999f-8f93-4fd5-95c5-de576ab93b00 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:32.106070 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c999f-8f93-4fd5-95c5-de576ab93b00 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:32.106403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c999f-8f93-4fd5-95c5-de576ab93b00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:32.132610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c04c999f-8f93-4fd5-95c5-de576ab93b00 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:32.134998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c04c999f-8f93-4fd5-95c5-de576ab93b00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:32.150162 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648afff6-2fd6-46a1-991e-d6ce50e9bad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:32.151161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648afff6-2fd6-46a1-991e-d6ce50e9bad0 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:32.151588 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648afff6-2fd6-46a1-991e-d6ce50e9bad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:32.152006 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648afff6-2fd6-46a1-991e-d6ce50e9bad0 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:32.153074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648afff6-2fd6-46a1-991e-d6ce50e9bad0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:32.155039 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648afff6-2fd6-46a1-991e-d6ce50e9bad0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:32.195657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 18/37] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 15:12:31 2022] POST /identity/v3/projects => generated 301 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:12:32.196527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648afff6-2fd6-46a1-991e-d6ce50e9bad0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:32.244281 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 20/38] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:12:32.270621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836fccc-7b0e-46e6-8df6-9467dd3c48ef None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:32.270996 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836fccc-7b0e-46e6-8df6-9467dd3c48ef None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:32.271308 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a836fccc-7b0e-46e6-8df6-9467dd3c48ef None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:32.273824 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 21/39] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:12:32.280699 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600f7aa-9069-4e2e-b0f7-5e290050cf93 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:32.281675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600f7aa-9069-4e2e-b0f7-5e290050cf93 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:32.281675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f600f7aa-9069-4e2e-b0f7-5e290050cf93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:32.303962 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f600f7aa-9069-4e2e-b0f7-5e290050cf93 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:32.305507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f600f7aa-9069-4e2e-b0f7-5e290050cf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:32.310810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:32.360434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648afff6-2fd6-46a1-991e-d6ce50e9bad0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:32.363011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:32.394118 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 22/40] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:12:32.405360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360230c4-24ec-429b-90f3-ea7084cf70e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:32.405998 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360230c4-24ec-429b-90f3-ea7084cf70e4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:32.406302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360230c4-24ec-429b-90f3-ea7084cf70e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:32.406577 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360230c4-24ec-429b-90f3-ea7084cf70e4 None admin] PATH_INFO: `/v3/users/793b87d316d544f9aecda76f798927dc` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:32.407941 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360230c4-24ec-429b-90f3-ea7084cf70e4 None admin] RBAC: Authorizing `identity:get_user(user_id=793b87d316d544f9aecda76f798927dc)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:32.409649 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360230c4-24ec-429b-90f3-ea7084cf70e4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:32.411421 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 19/41] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:32 2022] GET /identity/v3/users/793b87d316d544f9aecda76f798927dc => generated 247 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:32.417740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600f7aa-9069-4e2e-b0f7-5e290050cf93 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:32.521580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360230c4-24ec-429b-90f3-ea7084cf70e4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:32.527819 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f600f7aa-9069-4e2e-b0f7-5e290050cf93 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:32.569168 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e9fac-8ebb-4f67-88c7-58827a49a155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:32.570445 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e9fac-8ebb-4f67-88c7-58827a49a155 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:32.570918 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e9fac-8ebb-4f67-88c7-58827a49a155 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:32.571057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6e9fac-8ebb-4f67-88c7-58827a49a155 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:32.572314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e9fac-8ebb-4f67-88c7-58827a49a155 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:32.573716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6e9fac-8ebb-4f67-88c7-58827a49a155 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:32.575781 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 23/42] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:12:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:32.581392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e9fac-8ebb-4f67-88c7-58827a49a155 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:32.637147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:32.640296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360230c4-24ec-429b-90f3-ea7084cf70e4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:32.676823 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f68fbb-ea9c-436c-8b43-1de8873de7c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:32.677449 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f68fbb-ea9c-436c-8b43-1de8873de7c9 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:32.677733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f68fbb-ea9c-436c-8b43-1de8873de7c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:32.678010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f68fbb-ea9c-436c-8b43-1de8873de7c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:32.678728 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f68fbb-ea9c-436c-8b43-1de8873de7c9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:32.680686 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f68fbb-ea9c-436c-8b43-1de8873de7c9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:32.690043 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:32.696796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6e9fac-8ebb-4f67-88c7-58827a49a155 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:32.697808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 20/43] 10.222.0.76 () {60 vars in 1131 bytes} [Wed Aug 24 15:12:32 2022] POST /identity/v3/roles => generated 223 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:12:32.699850 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f68fbb-ea9c-436c-8b43-1de8873de7c9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:32.740026 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac6a32-79d2-4ac9-a3f8-d74e6e890f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:32.740720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac6a32-79d2-4ac9-a3f8-d74e6e890f43 None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:32.741001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac6a32-79d2-4ac9-a3f8-d74e6e890f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:32.741285 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac6a32-79d2-4ac9-a3f8-d74e6e890f43 None admin] PATH_INFO: `/v3/domains/default/users/793b87d316d544f9aecda76f798927dc/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:32.744061 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac6a32-79d2-4ac9-a3f8-d74e6e890f43 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=793b87d316d544f9aecda76f798927dc, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:32.745565 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ac6a32-79d2-4ac9-a3f8-d74e6e890f43 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:32.761160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 24/44] 10.222.0.76 () {58 vars in 1361 bytes} [Wed Aug 24 15:12:32 2022] PUT /identity/v3/domains/default/users/793b87d316d544f9aecda76f798927dc/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:12:32.823647 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f68fbb-ea9c-436c-8b43-1de8873de7c9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:32.867429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcebcaa-f193-4c02-a466-f87ed04c3725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:32.868079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcebcaa-f193-4c02-a466-f87ed04c3725 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:32.868359 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcebcaa-f193-4c02-a466-f87ed04c3725 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:32.868629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcebcaa-f193-4c02-a466-f87ed04c3725 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:32.869800 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcebcaa-f193-4c02-a466-f87ed04c3725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:32.871315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcebcaa-f193-4c02-a466-f87ed04c3725 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:32.873124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 21/45] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:12:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:32.878799 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac6a32-79d2-4ac9-a3f8-d74e6e890f43 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:33.098327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac6a32-79d2-4ac9-a3f8-d74e6e890f43 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:33.100646 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:33.138048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d295268-5cdf-47b0-9f45-22314413f4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:33.138781 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d295268-5cdf-47b0-9f45-22314413f4c7 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:33.139134 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d295268-5cdf-47b0-9f45-22314413f4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:33.139457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d295268-5cdf-47b0-9f45-22314413f4c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:33.140220 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d295268-5cdf-47b0-9f45-22314413f4c7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:33.141864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d295268-5cdf-47b0-9f45-22314413f4c7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:33.170631 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 25/46] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 15:12:32 2022] POST /identity/v3/projects => generated 312 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:12:34.739935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de0329-9dd7-4b56-a41d-ada8cf525479 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:34.740449 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de0329-9dd7-4b56-a41d-ada8cf525479 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:34.740552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10de0329-9dd7-4b56-a41d-ada8cf525479 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:34.742403 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 22/47] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:34.748072 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af59c7-4430-49c9-a4e6-b0e31897573c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:34.748496 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af59c7-4430-49c9-a4e6-b0e31897573c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:34.748623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8af59c7-4430-49c9-a4e6-b0e31897573c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:34.767103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b8af59c7-4430-49c9-a4e6-b0e31897573c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:34.768302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b8af59c7-4430-49c9-a4e6-b0e31897573c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:34.836848 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 26/48] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:35.116436 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de0329-9dd7-4b56-a41d-ada8cf525479 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:35.121475 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0091c-e202-48aa-b291-a66904813fe9 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:35.121759 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0091c-e202-48aa-b291-a66904813fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:35.122033 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0091c-e202-48aa-b291-a66904813fe9 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:35.123415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 27/49] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:12:35.128290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706985b-21aa-417e-9eb7-947a3b735d8d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:35.128573 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706985b-21aa-417e-9eb7-947a3b735d8d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:35.128848 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1706985b-21aa-417e-9eb7-947a3b735d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:35.146976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-1706985b-21aa-417e-9eb7-947a3b735d8d None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:35.148742 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1706985b-21aa-417e-9eb7-947a3b735d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:35.183484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:35.219896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 28/50] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:12:35.221988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10de0329-9dd7-4b56-a41d-ada8cf525479 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:35.256598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646f00a-7371-4679-a538-a46f92369573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:35.257278 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646f00a-7371-4679-a538-a46f92369573 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:35.258055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646f00a-7371-4679-a538-a46f92369573 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:35.258407 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a646f00a-7371-4679-a538-a46f92369573 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:35.269949 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a646f00a-7371-4679-a538-a46f92369573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:35.276116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a646f00a-7371-4679-a538-a46f92369573 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:12:35.276116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:35.276116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:35.276116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:35.276116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:35.276116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:35.276116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:35.276116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:35.276116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:35.276116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:35.276116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:35.276116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:35.276116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:35.276116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:35.276116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:35.276116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:35.277170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:35.277170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:35.277170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:35.277170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:35.277170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:35.277170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:35.277170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:35.277170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:35.277170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:35.277170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:35.277170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:35.277170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:35.277170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:35.277170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:35.277170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:35.277170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:35.277929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:35.277929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:35.277929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:35.277929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:35.277929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:35.277929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:35.277929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:35.277929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:35.277929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:35.277929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:35.277929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:12:35.277929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:12:35.277929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 23/51] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:12:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:35.283112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706985b-21aa-417e-9eb7-947a3b735d8d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:35.301362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:35.386332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1706985b-21aa-417e-9eb7-947a3b735d8d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:35.419630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4b7f0-90a8-47d1-80d6-bbb28441268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:35.420239 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4b7f0-90a8-47d1-80d6-bbb28441268b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:35.420513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4b7f0-90a8-47d1-80d6-bbb28441268b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:35.420782 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f4b7f0-90a8-47d1-80d6-bbb28441268b None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:35.421444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4b7f0-90a8-47d1-80d6-bbb28441268b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:35.421749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4b7f0-90a8-47d1-80d6-bbb28441268b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:35.423171 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f4b7f0-90a8-47d1-80d6-bbb28441268b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:35.431645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 29/52] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:12:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:35.437279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646f00a-7371-4679-a538-a46f92369573 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:35.500722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4b7f0-90a8-47d1-80d6-bbb28441268b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:35.541254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a646f00a-7371-4679-a538-a46f92369573 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:35.568013 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:35.577308 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05f855-320a-4017-9c21-bf60bda55d00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:35.577983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05f855-320a-4017-9c21-bf60bda55d00 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:35.578288 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05f855-320a-4017-9c21-bf60bda55d00 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:35.578566 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a05f855-320a-4017-9c21-bf60bda55d00 None admin] PATH_INFO: `/v3/users/793b87d316d544f9aecda76f798927dc` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:35.579705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05f855-320a-4017-9c21-bf60bda55d00 None admin] RBAC: Authorizing `identity:get_user(user_id=793b87d316d544f9aecda76f798927dc)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:35.581165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a05f855-320a-4017-9c21-bf60bda55d00 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:35.582979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 24/53] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:35 2022] GET /identity/v3/users/793b87d316d544f9aecda76f798927dc => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:35.586621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:35.588577 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05f855-320a-4017-9c21-bf60bda55d00 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:35.611047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f4b7f0-90a8-47d1-80d6-bbb28441268b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:35.645446 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98d519-0447-478d-b85c-665d22e0ada9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:35.646004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98d519-0447-478d-b85c-665d22e0ada9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:35.646316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98d519-0447-478d-b85c-665d22e0ada9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:35.646593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d98d519-0447-478d-b85c-665d22e0ada9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:35.647711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98d519-0447-478d-b85c-665d22e0ada9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:35.649204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d98d519-0447-478d-b85c-665d22e0ada9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:35.651048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 30/54] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:12:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:12:35.656415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98d519-0447-478d-b85c-665d22e0ada9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:35.690066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a05f855-320a-4017-9c21-bf60bda55d00 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:35.723037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae50a77-9d79-46e5-8487-82d7648dffb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:35.723666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae50a77-9d79-46e5-8487-82d7648dffb3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:35.723946 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae50a77-9d79-46e5-8487-82d7648dffb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:35.724218 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae50a77-9d79-46e5-8487-82d7648dffb3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:35.725330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae50a77-9d79-46e5-8487-82d7648dffb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:35.727294 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae50a77-9d79-46e5-8487-82d7648dffb3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:35.728852 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 25/55] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:12:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:12:35.734553 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae50a77-9d79-46e5-8487-82d7648dffb3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:35.757089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d98d519-0447-478d-b85c-665d22e0ada9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:35.792167 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c250faf4-a126-44ee-ac95-2c68f8a36060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:35.792780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250faf4-a126-44ee-ac95-2c68f8a36060 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:35.793065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250faf4-a126-44ee-ac95-2c68f8a36060 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:35.793339 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c250faf4-a126-44ee-ac95-2c68f8a36060 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:35.793978 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250faf4-a126-44ee-ac95-2c68f8a36060 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:35.795460 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c250faf4-a126-44ee-ac95-2c68f8a36060 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:35.805752 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:35.816940 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:35.824867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 31/56] 10.222.0.76 () {60 vars in 1140 bytes} [Wed Aug 24 15:12:35 2022] POST /identity/v3/projects => generated 298 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:12:35.848543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae50a77-9d79-46e5-8487-82d7648dffb3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:35.886137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e93a468-2aa1-4256-b328-b22ca1cbbbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:35.886813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93a468-2aa1-4256-b328-b22ca1cbbbad None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:35.887098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93a468-2aa1-4256-b328-b22ca1cbbbad None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:35.887377 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e93a468-2aa1-4256-b328-b22ca1cbbbad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:35.888135 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93a468-2aa1-4256-b328-b22ca1cbbbad None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.domain.id=default, user.id=793b87d316d544f9aecda76f798927dc) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:35.888449 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93a468-2aa1-4256-b328-b22ca1cbbbad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:35.889978 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e93a468-2aa1-4256-b328-b22ca1cbbbad None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:35.901447 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 26/57] 10.222.0.76 () {56 vars in 1322 bytes} [Wed Aug 24 15:12:35 2022] GET /identity/v3/role_assignments?user.id=793b87d316d544f9aecda76f798927dc&scope.domain.id=default&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 530 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:12:37.717964 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0824704-ac82-454b-8006-7bada2221d8d None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:37.718545 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0824704-ac82-454b-8006-7bada2221d8d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:37.718545 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0824704-ac82-454b-8006-7bada2221d8d None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:37.719891 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 32/58] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:37.724947 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523658-7ee6-4af2-9ea4-7a5cb66d721f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:37.725332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523658-7ee6-4af2-9ea4-7a5cb66d721f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:37.725504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48523658-7ee6-4af2-9ea4-7a5cb66d721f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:37.857459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-48523658-7ee6-4af2-9ea4-7a5cb66d721f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:37.858978 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-48523658-7ee6-4af2-9ea4-7a5cb66d721f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:37.924712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 27/59] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:38.189770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0824704-ac82-454b-8006-7bada2221d8d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:38.289855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0824704-ac82-454b-8006-7bada2221d8d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:38.322769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970c620-56d7-4270-97d2-175f434f69c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:38.323379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970c620-56d7-4270-97d2-175f434f69c3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:38.323654 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970c620-56d7-4270-97d2-175f434f69c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:38.323921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970c620-56d7-4270-97d2-175f434f69c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:38.325006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970c620-56d7-4270-97d2-175f434f69c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:38.326461 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970c620-56d7-4270-97d2-175f434f69c3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:38.328010 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 33/60] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:12:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:38.333638 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48523658-7ee6-4af2-9ea4-7a5cb66d721f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:38.387655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:38.430379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48523658-7ee6-4af2-9ea4-7a5cb66d721f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:38.463485 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c45d464-eb75-47ac-9790-5e6e0fc23f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:38.464032 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45d464-eb75-47ac-9790-5e6e0fc23f9d None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:38.464312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45d464-eb75-47ac-9790-5e6e0fc23f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:38.464585 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c45d464-eb75-47ac-9790-5e6e0fc23f9d None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:38.465232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45d464-eb75-47ac-9790-5e6e0fc23f9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:38.466706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c45d464-eb75-47ac-9790-5e6e0fc23f9d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:38.493024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 28/61] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:12:38 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:12:41.466851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bedd9-8de2-4898-968a-c336f5d2f5b2 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:41.467393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bedd9-8de2-4898-968a-c336f5d2f5b2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:41.467393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0bedd9-8de2-4898-968a-c336f5d2f5b2 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:41.468717 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 34/62] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:41.474039 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58820ae0-d225-4a8b-8c99-7b16b22a9061 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:41.474863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58820ae0-d225-4a8b-8c99-7b16b22a9061 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:41.475343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58820ae0-d225-4a8b-8c99-7b16b22a9061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:41.499172 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-58820ae0-d225-4a8b-8c99-7b16b22a9061 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:41.500886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-58820ae0-d225-4a8b-8c99-7b16b22a9061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:41.595434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 29/63] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:12:41.758818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e36fd-20c2-4050-b682-e430892677db None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:41.759353 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e36fd-20c2-4050-b682-e430892677db None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:41.759795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1e36fd-20c2-4050-b682-e430892677db None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:41.761294 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 35/64] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:41.779211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3281a-be73-437b-b254-19ed5864bc90 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:41.779779 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3281a-be73-437b-b254-19ed5864bc90 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:41.780234 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3281a-be73-437b-b254-19ed5864bc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:41.795341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e84ab3-4c0e-4e7f-8afd-b4b8c8023bdd None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:41.795934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e84ab3-4c0e-4e7f-8afd-b4b8c8023bdd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:41.796393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e84ab3-4c0e-4e7f-8afd-b4b8c8023bdd None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:41.797929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 36/65] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:12:41.802987 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f45d4-0a20-47b4-ac83-cdf0693e4b22 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:41.803599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f45d4-0a20-47b4-ac83-cdf0693e4b22 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:41.804159 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f45d4-0a20-47b4-ac83-cdf0693e4b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:41.822205 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4da3281a-be73-437b-b254-19ed5864bc90 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:41.825676 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4da3281a-be73-437b-b254-19ed5864bc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:41.838335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0f6f45d4-0a20-47b4-ac83-cdf0693e4b22 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:41.839839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0f6f45d4-0a20-47b4-ac83-cdf0693e4b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:41.894130 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:41.924659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:41.934000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 37/66] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:12:41.950651 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 30/67] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:42.036395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d6f27-2723-4e39-b36c-646ec08d0cd3 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:42.036942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d6f27-2723-4e39-b36c-646ec08d0cd3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:42.037405 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79d6f27-2723-4e39-b36c-646ec08d0cd3 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:42.039521 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 38/68] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:42.044460 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d6f27-2723-4e39-b36c-646ec08d0cd3 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:42.045025 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7b68e-2d03-4efb-9381-b8017faf132a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:42.045332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7b68e-2d03-4efb-9381-b8017faf132a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:42.045710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e7b68e-2d03-4efb-9381-b8017faf132a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:42.064455 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-89e7b68e-2d03-4efb-9381-b8017faf132a None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:42.066065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-89e7b68e-2d03-4efb-9381-b8017faf132a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:42.144679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 31/69] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:42.154435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57fb9e-fd5b-446e-be55-c44a1e649b67 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:42.154435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57fb9e-fd5b-446e-be55-c44a1e649b67 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:42.154435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b57fb9e-fd5b-446e-be55-c44a1e649b67 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:42.154435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 32/70] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:42.156455 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18028e83-da82-4b68-b06a-e4a890fab510 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:42.156740 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18028e83-da82-4b68-b06a-e4a890fab510 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:42.157011 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18028e83-da82-4b68-b06a-e4a890fab510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:42.192536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-18028e83-da82-4b68-b06a-e4a890fab510 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:42.193735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-18028e83-da82-4b68-b06a-e4a890fab510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:42.224661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79d6f27-2723-4e39-b36c-646ec08d0cd3 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:42.265596 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395380d9-410a-4adc-bb5f-33ccb8da33a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:42.266552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395380d9-410a-4adc-bb5f-33ccb8da33a4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:42.267126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395380d9-410a-4adc-bb5f-33ccb8da33a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:42.267126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395380d9-410a-4adc-bb5f-33ccb8da33a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:42.279231 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395380d9-410a-4adc-bb5f-33ccb8da33a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:42.284034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-395380d9-410a-4adc-bb5f-33ccb8da33a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:12:42.284034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:42.284034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:42.284034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:42.284034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:42.284034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:42.284034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:42.284034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:42.284034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:42.284034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:42.284034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:42.284034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:42.284034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:42.284034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:42.284034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:42.284034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:42.284853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:42.284853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:42.284853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:42.284853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:42.284853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:42.284853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:42.284853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:42.284853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:42.284853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:42.284853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:42.284853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:42.284853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:42.284853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:42.284853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:42.284853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:42.284853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:42.285627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:42.285627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:42.285627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:42.285627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:42.285627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:42.285627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:42.285627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:42.285627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:42.285627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:42.285627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:42.285627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:12:42.285627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:12:42.285627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 39/71] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:12:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:12:42.286844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395380d9-410a-4adc-bb5f-33ccb8da33a4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:42.296793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 33/72] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:12:42.299140 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18028e83-da82-4b68-b06a-e4a890fab510 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:42.340410 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:42.348384 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:42.397308 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395380d9-410a-4adc-bb5f-33ccb8da33a4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:42.425236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18028e83-da82-4b68-b06a-e4a890fab510 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:42.436885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f0a18-6156-4baa-9cd6-7aa665d8098c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:42.437522 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f0a18-6156-4baa-9cd6-7aa665d8098c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:42.437856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f0a18-6156-4baa-9cd6-7aa665d8098c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:42.438280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f0a18-6156-4baa-9cd6-7aa665d8098c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:42.449482 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f0a18-6156-4baa-9cd6-7aa665d8098c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:42.454323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-f83f0a18-6156-4baa-9cd6-7aa665d8098c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:12:42.454323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:42.454323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:42.454323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:42.454323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:42.454323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:42.454323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:42.454323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:42.454323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:42.454323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:42.454323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:42.454323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:42.454323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:42.454323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:42.454323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:42.454323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:42.455180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:42.455180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:42.455180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:42.455180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:42.455180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:42.455180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:42.455180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:42.455180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:42.455180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:42.455180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:42.455180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:42.455180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:42.455180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:42.455180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:42.455180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:42.455180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:42.455924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:42.455924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:42.455924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:42.455924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:42.455924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:42.455924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:42.455924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:42.455924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:42.455924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:42.455924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:42.455924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:12:42.455924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:12:42.455924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 40/73] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:12:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:42.456550 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f0a18-6156-4baa-9cd6-7aa665d8098c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:42.464982 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974aafe-cb51-4c68-a786-6f00b61fea6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:42.465623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974aafe-cb51-4c68-a786-6f00b61fea6d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:42.465904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974aafe-cb51-4c68-a786-6f00b61fea6d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:42.466176 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974aafe-cb51-4c68-a786-6f00b61fea6d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:42.480128 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974aafe-cb51-4c68-a786-6f00b61fea6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:42.485458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5974aafe-cb51-4c68-a786-6f00b61fea6d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:12:42.485458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:42.485458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:42.485458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:42.485458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:42.485458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:42.485458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:42.485458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:42.485458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:42.485458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:42.485458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:42.485458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:42.485458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:42.485458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:42.485458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:42.485458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:42.486322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:42.486322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:42.486322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:42.486322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:42.486322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:42.486322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:42.486322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:42.486322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:42.486322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:42.486322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:42.486322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:42.486322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:42.486322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:42.486322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:42.486322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:42.486322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:42.487065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:42.487065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:42.487065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:42.487065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:42.487065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:42.487065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:42.487065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:42.487065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:42.487065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:42.487065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:42.487065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:12:42.487065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:12:42.487065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 34/74] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:12:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:42.487677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974aafe-cb51-4c68-a786-6f00b61fea6d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:42.584783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f0a18-6156-4baa-9cd6-7aa665d8098c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:42.587290 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:42.596634 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974aafe-cb51-4c68-a786-6f00b61fea6d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:42.602932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:42.622477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74736c02-5dd0-4901-9da6-f59c0419fb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:42.623093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74736c02-5dd0-4901-9da6-f59c0419fb94 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:42.623374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74736c02-5dd0-4901-9da6-f59c0419fb94 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:42.623679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74736c02-5dd0-4901-9da6-f59c0419fb94 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:42.624466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74736c02-5dd0-4901-9da6-f59c0419fb94 None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:42.624767 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74736c02-5dd0-4901-9da6-f59c0419fb94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:42.626179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74736c02-5dd0-4901-9da6-f59c0419fb94 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:42.631849 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade13848-0d59-456f-bae1-a581d4b8d676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:42.632512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade13848-0d59-456f-bae1-a581d4b8d676 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:42.632798 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade13848-0d59-456f-bae1-a581d4b8d676 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:42.633071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade13848-0d59-456f-bae1-a581d4b8d676 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:42.634205 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade13848-0d59-456f-bae1-a581d4b8d676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:42.634702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 41/75] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:12:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:42.635514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74736c02-5dd0-4901-9da6-f59c0419fb94 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:42.635694 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade13848-0d59-456f-bae1-a581d4b8d676 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:42.637390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 35/76] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:12:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:42.638074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade13848-0d59-456f-bae1-a581d4b8d676 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:42.738934 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade13848-0d59-456f-bae1-a581d4b8d676 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:42.739646 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74736c02-5dd0-4901-9da6-f59c0419fb94 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:42.773191 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245a98f-a3cd-4c30-9904-c3df9346485e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:42.773435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e0d50-8c0f-4f5e-9163-292e79a33a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:42.773883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e0d50-8c0f-4f5e-9163-292e79a33a9e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:42.773973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245a98f-a3cd-4c30-9904-c3df9346485e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:42.774165 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e0d50-8c0f-4f5e-9163-292e79a33a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:42.774290 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245a98f-a3cd-4c30-9904-c3df9346485e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:42.774465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0e0d50-8c0f-4f5e-9163-292e79a33a9e None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:42.774550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e245a98f-a3cd-4c30-9904-c3df9346485e None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:42.775393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e0d50-8c0f-4f5e-9163-292e79a33a9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:42.775503 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245a98f-a3cd-4c30-9904-c3df9346485e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:42.775695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e0d50-8c0f-4f5e-9163-292e79a33a9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:42.775780 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245a98f-a3cd-4c30-9904-c3df9346485e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:42.777080 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e245a98f-a3cd-4c30-9904-c3df9346485e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:42.777164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0e0d50-8c0f-4f5e-9163-292e79a33a9e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:42.785603 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 42/77] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:12:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:42.786455 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e0d50-8c0f-4f5e-9163-292e79a33a9e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:42.789807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 36/78] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:12:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:42.790743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245a98f-a3cd-4c30-9904-c3df9346485e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:42.815133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:42.833331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:42.891872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e245a98f-a3cd-4c30-9904-c3df9346485e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:42.892489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0e0d50-8c0f-4f5e-9163-292e79a33a9e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:42.927012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec02e86-0571-403d-9b26-e49a16bf72dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:42.927643 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec02e86-0571-403d-9b26-e49a16bf72dc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:42.927744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8648d-f031-483f-9478-8b8499860480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:42.927925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec02e86-0571-403d-9b26-e49a16bf72dc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:42.928196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec02e86-0571-403d-9b26-e49a16bf72dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:42.928324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8648d-f031-483f-9478-8b8499860480 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:42.928602 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8648d-f031-483f-9478-8b8499860480 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:42.928903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8648d-f031-483f-9478-8b8499860480 None admin] PATH_INFO: `/v3/users/179374916edf4dd480273f6c8c748ff6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:42.938280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec02e86-0571-403d-9b26-e49a16bf72dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:42.943506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-8ec02e86-0571-403d-9b26-e49a16bf72dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:12:42.943506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:42.943506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:42.943506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:42.943506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:42.943506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:42.943506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:42.943506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:42.943506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:42.943506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:42.943506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:42.943506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:42.943506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:42.943506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:42.943506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:42.943506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:42.944624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:42.944624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:42.944624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:42.944624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:42.944624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:42.944624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:42.944624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:42.944624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:42.944624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:42.944624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:42.944624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:42.944624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:42.944624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:42.944624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:42.944624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:42.944624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:42.945503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:42.945503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:42.945503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:42.945503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:42.945503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:42.945503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:42.945503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:42.945503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:42.945503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:42.945503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:42.945503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:12:42.945503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:12:42.945503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 43/79] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:12:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:42.945503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec02e86-0571-403d-9b26-e49a16bf72dc None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:42.947092 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8648d-f031-483f-9478-8b8499860480 None admin] RBAC: Authorizing `identity:get_user(user_id=179374916edf4dd480273f6c8c748ff6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:42.948579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e8648d-f031-483f-9478-8b8499860480 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:42.950540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 37/80] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:42 2022] GET /identity/v3/users/179374916edf4dd480273f6c8c748ff6 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:42.951307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8648d-f031-483f-9478-8b8499860480 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:43.032732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:43.045036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:43.048528 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec02e86-0571-403d-9b26-e49a16bf72dc None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:43.053215 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8648d-f031-483f-9478-8b8499860480 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:43.082066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ec045-7307-4b03-9141-1776a5076619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:43.082705 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ec045-7307-4b03-9141-1776a5076619 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:43.082986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ec045-7307-4b03-9141-1776a5076619 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:43.083260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77ec045-7307-4b03-9141-1776a5076619 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:43.083904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ec045-7307-4b03-9141-1776a5076619 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:43.085840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77ec045-7307-4b03-9141-1776a5076619 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:43.086891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d655ee-7849-4ae5-ae04-ba68dd48082e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:43.087527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d655ee-7849-4ae5-ae04-ba68dd48082e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:43.088272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d655ee-7849-4ae5-ae04-ba68dd48082e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:43.088548 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d655ee-7849-4ae5-ae04-ba68dd48082e None admin] PATH_INFO: `/v3/users/793b87d316d544f9aecda76f798927dc` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:43.089684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d655ee-7849-4ae5-ae04-ba68dd48082e None admin] RBAC: Authorizing `identity:get_user(user_id=793b87d316d544f9aecda76f798927dc)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:43.091151 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d655ee-7849-4ae5-ae04-ba68dd48082e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:43.092734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 38/81] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:42 2022] GET /identity/v3/users/793b87d316d544f9aecda76f798927dc => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:43.093498 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d655ee-7849-4ae5-ae04-ba68dd48082e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:43.115959 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 44/82] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:12:42 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:12:43.117166 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ec045-7307-4b03-9141-1776a5076619 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:43.215924 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d655ee-7849-4ae5-ae04-ba68dd48082e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:43.235658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77ec045-7307-4b03-9141-1776a5076619 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:43.256953 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc833e69-6075-4e95-afd5-3892c68d7443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:43.258193 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc833e69-6075-4e95-afd5-3892c68d7443 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:43.258508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc833e69-6075-4e95-afd5-3892c68d7443 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:43.258783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc833e69-6075-4e95-afd5-3892c68d7443 None admin] PATH_INFO: `/v3/users/179374916edf4dd480273f6c8c748ff6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:43.260488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc833e69-6075-4e95-afd5-3892c68d7443 None admin] RBAC: Authorizing `identity:get_user(user_id=179374916edf4dd480273f6c8c748ff6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:43.262068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc833e69-6075-4e95-afd5-3892c68d7443 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:43.264069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 39/83] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:43 2022] GET /identity/v3/users/179374916edf4dd480273f6c8c748ff6 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:43.264867 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc833e69-6075-4e95-afd5-3892c68d7443 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:43.286166 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:43.290954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d5b7e-0bf9-4562-8d54-4493a092cf05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:43.291579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d5b7e-0bf9-4562-8d54-4493a092cf05 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:43.291868 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d5b7e-0bf9-4562-8d54-4493a092cf05 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:43.292144 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19d5b7e-0bf9-4562-8d54-4493a092cf05 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:43.292839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d5b7e-0bf9-4562-8d54-4493a092cf05 None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:43.293146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d5b7e-0bf9-4562-8d54-4493a092cf05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:43.294620 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19d5b7e-0bf9-4562-8d54-4493a092cf05 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:43.305637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 45/84] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:12:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:43.306626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d5b7e-0bf9-4562-8d54-4493a092cf05 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:43.330645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:43.384794 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc833e69-6075-4e95-afd5-3892c68d7443 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:43.432607 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2abb6-0326-47ad-8d08-48e894b504e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:43.433555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2abb6-0326-47ad-8d08-48e894b504e5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:43.434126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2abb6-0326-47ad-8d08-48e894b504e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:43.434591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a2abb6-0326-47ad-8d08-48e894b504e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:43.441524 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19d5b7e-0bf9-4562-8d54-4493a092cf05 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:43.485091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cba5d-9827-49df-9dc9-fa226dcec55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:43.485920 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cba5d-9827-49df-9dc9-fa226dcec55e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:43.486371 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cba5d-9827-49df-9dc9-fa226dcec55e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:43.486806 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cba5d-9827-49df-9dc9-fa226dcec55e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:43.528197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2abb6-0326-47ad-8d08-48e894b504e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:43.529850 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a2abb6-0326-47ad-8d08-48e894b504e5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:43.553561 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:43.569398 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cba5d-9827-49df-9dc9-fa226dcec55e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:43.571823 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cba5d-9827-49df-9dc9-fa226dcec55e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:43.592021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:43.665120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 40/85] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:12:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:12:43.666387 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2abb6-0326-47ad-8d08-48e894b504e5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:43.689788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 46/86] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:12:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:12:43.690686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cba5d-9827-49df-9dc9-fa226dcec55e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:43.794553 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a2abb6-0326-47ad-8d08-48e894b504e5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:43.812871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:43.813122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cba5d-9827-49df-9dc9-fa226dcec55e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:43.833268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6c8e8-c10a-4c89-a516-d7fa86e515ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:43.833926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6c8e8-c10a-4c89-a516-d7fa86e515ab None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:43.834151 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6c8e8-c10a-4c89-a516-d7fa86e515ab None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:43.834453 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e6c8e8-c10a-4c89-a516-d7fa86e515ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:43.838512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:43.867301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3295ce-2cd6-4781-993f-a4fbcd0725df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:43.868272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3295ce-2cd6-4781-993f-a4fbcd0725df None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:43.868826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3295ce-2cd6-4781-993f-a4fbcd0725df None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:43.869253 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3295ce-2cd6-4781-993f-a4fbcd0725df None admin] PATH_INFO: `/v3/users/179374916edf4dd480273f6c8c748ff6` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:43.870580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3295ce-2cd6-4781-993f-a4fbcd0725df None admin] RBAC: Authorizing `identity:get_user(user_id=179374916edf4dd480273f6c8c748ff6)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:43.872500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3295ce-2cd6-4781-993f-a4fbcd0725df None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:43.874378 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 47/87] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:43 2022] GET /identity/v3/users/179374916edf4dd480273f6c8c748ff6 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:43.875450 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3295ce-2cd6-4781-993f-a4fbcd0725df None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:43.912612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6c8e8-c10a-4c89-a516-d7fa86e515ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:43.914117 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e6c8e8-c10a-4c89-a516-d7fa86e515ab None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:43.988333 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3295ce-2cd6-4781-993f-a4fbcd0725df None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:44.017112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 41/88] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:12:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:12:44.021796 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6c8e8-c10a-4c89-a516-d7fa86e515ab None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:44.033039 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925ac18-cfb6-4b4e-8c59-fa4a09d3254c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:44.034032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925ac18-cfb6-4b4e-8c59-fa4a09d3254c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:44.034580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925ac18-cfb6-4b4e-8c59-fa4a09d3254c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:44.034994 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9925ac18-cfb6-4b4e-8c59-fa4a09d3254c None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:44.036299 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925ac18-cfb6-4b4e-8c59-fa4a09d3254c None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:44.037137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:44.039247 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9925ac18-cfb6-4b4e-8c59-fa4a09d3254c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:44.041922 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 48/89] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:43 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:44.042927 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925ac18-cfb6-4b4e-8c59-fa4a09d3254c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:44.104797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:44.126254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e6c8e8-c10a-4c89-a516-d7fa86e515ab None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:44.157172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9925ac18-cfb6-4b4e-8c59-fa4a09d3254c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:44.161798 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ce4c9-fe7f-4de6-9929-f51a21d4c3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:44.162473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ce4c9-fe7f-4de6-9929-f51a21d4c3fb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:44.162750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ce4c9-fe7f-4de6-9929-f51a21d4c3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:44.163021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ce4c9-fe7f-4de6-9929-f51a21d4c3fb None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:44.164212 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ce4c9-fe7f-4de6-9929-f51a21d4c3fb None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:44.165671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80ce4c9-fe7f-4de6-9929-f51a21d4c3fb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:44.167362 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 42/90] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:44 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:44.168131 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ce4c9-fe7f-4de6-9929-f51a21d4c3fb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:44.197914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b810ffc-5f99-4020-826f-8cca95d650c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:44.198802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b810ffc-5f99-4020-826f-8cca95d650c2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:44.199323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b810ffc-5f99-4020-826f-8cca95d650c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:44.199743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b810ffc-5f99-4020-826f-8cca95d650c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:44.255226 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:44.272402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ce4c9-fe7f-4de6-9929-f51a21d4c3fb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:44.279960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b810ffc-5f99-4020-826f-8cca95d650c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:44.281775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b810ffc-5f99-4020-826f-8cca95d650c2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:44.309735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee675604-f949-40f6-8221-89e33535057a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:44.310893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee675604-f949-40f6-8221-89e33535057a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:44.311039 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee675604-f949-40f6-8221-89e33535057a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:44.311290 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee675604-f949-40f6-8221-89e33535057a None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:44.312720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee675604-f949-40f6-8221-89e33535057a None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:44.314643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee675604-f949-40f6-8221-89e33535057a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:44.321957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 43/91] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:44 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:44.321957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee675604-f949-40f6-8221-89e33535057a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:44.352837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:44.418519 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 49/92] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:12:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:12:44.422587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b810ffc-5f99-4020-826f-8cca95d650c2 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:44.450073 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee675604-f949-40f6-8221-89e33535057a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:44.491087 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7f2b7-75b8-403a-8148-fbd39b6a25a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:44.492234 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7f2b7-75b8-403a-8148-fbd39b6a25a2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:44.492513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7f2b7-75b8-403a-8148-fbd39b6a25a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:44.492779 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7f2b7-75b8-403a-8148-fbd39b6a25a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:44.493786 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7f2b7-75b8-403a-8148-fbd39b6a25a2 None admin] RBAC: Adding query filter params (role.id=8d794ef8abb94809bd988c5db32c0037, scope.project.id=7e93b768826c4821ad3dd853154e62b4, user.id=179374916edf4dd480273f6c8c748ff6) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:44.494090 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7f2b7-75b8-403a-8148-fbd39b6a25a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:44.496195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f7f2b7-75b8-403a-8148-fbd39b6a25a2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:44.505530 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 44/93] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:12:44 2022] GET /identity/v3/role_assignments?user.id=179374916edf4dd480273f6c8c748ff6&scope.project.id=7e93b768826c4821ad3dd853154e62b4&role.id=8d794ef8abb94809bd988c5db32c0037 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:44.514411 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7f2b7-75b8-403a-8148-fbd39b6a25a2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:44.556236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:44.592213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b810ffc-5f99-4020-826f-8cca95d650c2 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:44.683610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05db39-f8d6-43d8-a1a9-d945445b4a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:44.684152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05db39-f8d6-43d8-a1a9-d945445b4a40 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:44.684434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05db39-f8d6-43d8-a1a9-d945445b4a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:44.684709 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05db39-f8d6-43d8-a1a9-d945445b4a40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:44.685559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05db39-f8d6-43d8-a1a9-d945445b4a40 None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=7e93b768826c4821ad3dd853154e62b4, user.id=793b87d316d544f9aecda76f798927dc) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:44.686038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05db39-f8d6-43d8-a1a9-d945445b4a40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:44.688501 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05db39-f8d6-43d8-a1a9-d945445b4a40 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:44.700060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f7f2b7-75b8-403a-8148-fbd39b6a25a2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:44.703615 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 50/94] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:12:44 2022] GET /identity/v3/role_assignments?user.id=793b87d316d544f9aecda76f798927dc&scope.project.id=7e93b768826c4821ad3dd853154e62b4&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 259 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:44.703615 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05db39-f8d6-43d8-a1a9-d945445b4a40 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:44.729712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:44.749822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9afa1a-34de-4243-8f12-c270696448b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:44.750764 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9afa1a-34de-4243-8f12-c270696448b9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:44.751245 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9afa1a-34de-4243-8f12-c270696448b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:44.751686 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9afa1a-34de-4243-8f12-c270696448b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:44.752580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9afa1a-34de-4243-8f12-c270696448b9 None admin] RBAC: Adding query filter params (role.id=d123411b649d4acb8648d0c0b80aae6d, scope.project.id=7e93b768826c4821ad3dd853154e62b4, user.id=179374916edf4dd480273f6c8c748ff6) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:44.753069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9afa1a-34de-4243-8f12-c270696448b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:44.754729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9afa1a-34de-4243-8f12-c270696448b9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:44.766250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 45/95] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:12:44 2022] GET /identity/v3/role_assignments?user.id=179374916edf4dd480273f6c8c748ff6&scope.project.id=7e93b768826c4821ad3dd853154e62b4&role.id=d123411b649d4acb8648d0c0b80aae6d => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:44.834188 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05db39-f8d6-43d8-a1a9-d945445b4a40 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:44.880387 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d65b1-b7d2-4fa1-9f62-2575628908e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:44.881067 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d65b1-b7d2-4fa1-9f62-2575628908e4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:44.881351 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d65b1-b7d2-4fa1-9f62-2575628908e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:44.881626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d65b1-b7d2-4fa1-9f62-2575628908e4 None admin] PATH_INFO: `/v3/projects/39c9b647649b472bb19adde410a53a68` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:44.883607 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d65b1-b7d2-4fa1-9f62-2575628908e4 None admin] RBAC: Authorizing `identity:get_project(project_id=39c9b647649b472bb19adde410a53a68)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:44.885208 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d65b1-b7d2-4fa1-9f62-2575628908e4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:44.887265 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 51/96] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:44 2022] GET /identity/v3/projects/39c9b647649b472bb19adde410a53a68 => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:44.893531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9afa1a-34de-4243-8f12-c270696448b9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:45.019947 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:45.045268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9afa1a-34de-4243-8f12-c270696448b9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:45.093621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db96998f-c900-4fe7-8da6-49cdeaa9d98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:45.094262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96998f-c900-4fe7-8da6-49cdeaa9d98e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:45.094554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96998f-c900-4fe7-8da6-49cdeaa9d98e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:45.094832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db96998f-c900-4fe7-8da6-49cdeaa9d98e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:45.095571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96998f-c900-4fe7-8da6-49cdeaa9d98e None admin] RBAC: Adding query filter params (role.id=8d794ef8abb94809bd988c5db32c0037, scope.project.id=39c9b647649b472bb19adde410a53a68, user.id=179374916edf4dd480273f6c8c748ff6) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:45.095884 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96998f-c900-4fe7-8da6-49cdeaa9d98e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:45.097386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db96998f-c900-4fe7-8da6-49cdeaa9d98e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:45.103808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 46/97] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:12:44 2022] GET /identity/v3/role_assignments?user.id=179374916edf4dd480273f6c8c748ff6&scope.project.id=39c9b647649b472bb19adde410a53a68&role.id=8d794ef8abb94809bd988c5db32c0037 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:45.446790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de013ec-b68a-412f-97be-a36680e5d415 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:45.447086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de013ec-b68a-412f-97be-a36680e5d415 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:45.447361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de013ec-b68a-412f-97be-a36680e5d415 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:45.448729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 52/98] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:45.454485 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ced70-4b6f-403c-b036-b49c65747c80 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:45.454777 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ced70-4b6f-403c-b036-b49c65747c80 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:45.455054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ced70-4b6f-403c-b036-b49c65747c80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:45.476348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-591ced70-4b6f-403c-b036-b49c65747c80 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:45.477547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-591ced70-4b6f-403c-b036-b49c65747c80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:45.492586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95706f5d-e0c0-4496-a7a0-64471f17a5d3 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:45.492878 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95706f5d-e0c0-4496-a7a0-64471f17a5d3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:45.493140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95706f5d-e0c0-4496-a7a0-64471f17a5d3 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:45.494575 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 53/99] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:12:45.506841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859a17e-e02b-4fbf-936f-5d8cd3603bb8 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:45.507732 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859a17e-e02b-4fbf-936f-5d8cd3603bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:45.508014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859a17e-e02b-4fbf-936f-5d8cd3603bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:45.533018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a859a17e-e02b-4fbf-936f-5d8cd3603bb8 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:45.536281 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a859a17e-e02b-4fbf-936f-5d8cd3603bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:45.604546 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 47/100] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:45.651124 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 54/101] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:12:45.991148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ced70-4b6f-403c-b036-b49c65747c80 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:46.085410 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:46.122605 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ced70-4b6f-403c-b036-b49c65747c80 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:46.168721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae873e-1625-4116-a7b6-3ab41fc44c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:46.169366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae873e-1625-4116-a7b6-3ab41fc44c7e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:46.169646 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae873e-1625-4116-a7b6-3ab41fc44c7e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:46.169919 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fae873e-1625-4116-a7b6-3ab41fc44c7e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:46.171104 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae873e-1625-4116-a7b6-3ab41fc44c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:46.172624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fae873e-1625-4116-a7b6-3ab41fc44c7e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:46.174508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 48/102] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:12:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:46.180835 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859a17e-e02b-4fbf-936f-5d8cd3603bb8 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:46.207704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:46.224058 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae873e-1625-4116-a7b6-3ab41fc44c7e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:46.349180 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fae873e-1625-4116-a7b6-3ab41fc44c7e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:46.392839 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffdd69-0017-42ee-9b8f-8f01ba1f7b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:46.393581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffdd69-0017-42ee-9b8f-8f01ba1f7b7e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:46.393866 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffdd69-0017-42ee-9b8f-8f01ba1f7b7e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:46.394137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ffdd69-0017-42ee-9b8f-8f01ba1f7b7e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:46.396083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859a17e-e02b-4fbf-936f-5d8cd3603bb8 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:46.412744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ffdd69-0017-42ee-9b8f-8f01ba1f7b7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:46.423177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-57ffdd69-0017-42ee-9b8f-8f01ba1f7b7e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:12:46.423177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:46.423177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:46.423177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:46.423177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:46.423177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:46.423177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:46.423177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:46.423177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:46.423177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:46.423177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:46.423177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:46.423177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:46.423177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:46.423177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:46.423177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:46.424065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:46.424065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:46.424065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:46.424065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:46.424065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:46.424065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:46.424065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:46.424065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:46.424065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:46.424065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:46.424065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:46.424065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:46.424065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:46.424065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:46.424065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:46.424065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:46.424809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:46.424809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:46.424809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:46.424809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:46.424809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:46.424809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:46.424809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:46.424809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:46.424809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:46.424809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:46.424809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:12:46.424809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:12:46.424809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 49/103] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:12:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:12:46.432144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffdd69-0017-42ee-9b8f-8f01ba1f7b7e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:46.438553 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:46.463485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ab501-1335-402b-a3f1-c0e80b1f43ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:46.464106 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ab501-1335-402b-a3f1-c0e80b1f43ff None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:46.464384 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ab501-1335-402b-a3f1-c0e80b1f43ff None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:46.464663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ab501-1335-402b-a3f1-c0e80b1f43ff None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:46.465320 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ab501-1335-402b-a3f1-c0e80b1f43ff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:46.466798 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41ab501-1335-402b-a3f1-c0e80b1f43ff None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:46.517899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 55/104] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 15:12:46 2022] POST /identity/v3/projects => generated 302 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:12:46.625023 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ffdd69-0017-42ee-9b8f-8f01ba1f7b7e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:46.675318 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c85e20-9dd8-4cb3-9903-ecf526dcb82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:46.675950 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c85e20-9dd8-4cb3-9903-ecf526dcb82e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:46.676231 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c85e20-9dd8-4cb3-9903-ecf526dcb82e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:46.676505 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c85e20-9dd8-4cb3-9903-ecf526dcb82e None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:46.677186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c85e20-9dd8-4cb3-9903-ecf526dcb82e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:46.677492 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c85e20-9dd8-4cb3-9903-ecf526dcb82e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:46.679488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c85e20-9dd8-4cb3-9903-ecf526dcb82e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:46.690423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 50/105] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:12:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:46.699408 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ab501-1335-402b-a3f1-c0e80b1f43ff None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:46.794791 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:46.833173 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ab501-1335-402b-a3f1-c0e80b1f43ff None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:46.877768 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcc6d1e-cf2c-4520-a1e6-314caa39a132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:46.878388 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc6d1e-cf2c-4520-a1e6-314caa39a132 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:46.878671 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc6d1e-cf2c-4520-a1e6-314caa39a132 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:46.878942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcc6d1e-cf2c-4520-a1e6-314caa39a132 None admin] PATH_INFO: `/v3/users/5878318c80a4470181cbe55d2c509015` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:46.898907 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc6d1e-cf2c-4520-a1e6-314caa39a132 None admin] RBAC: Authorizing `identity:get_user(user_id=5878318c80a4470181cbe55d2c509015)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:46.900644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcc6d1e-cf2c-4520-a1e6-314caa39a132 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:46.902528 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 56/106] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:46 2022] GET /identity/v3/users/5878318c80a4470181cbe55d2c509015 => generated 289 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:46.908140 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c85e20-9dd8-4cb3-9903-ecf526dcb82e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:47.021618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:47.076941 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c85e20-9dd8-4cb3-9903-ecf526dcb82e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:47.129303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb9797-dac9-4b24-aab6-eecaed6492c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:47.129842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb9797-dac9-4b24-aab6-eecaed6492c0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:47.130124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb9797-dac9-4b24-aab6-eecaed6492c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:47.130423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb9797-dac9-4b24-aab6-eecaed6492c0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:47.218672 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb9797-dac9-4b24-aab6-eecaed6492c0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:47.220374 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb9797-dac9-4b24-aab6-eecaed6492c0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:47.301264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:47.346939 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9b434-ae4b-401a-8574-7e91fd48318a None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:47.347556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9b434-ae4b-401a-8574-7e91fd48318a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:47.348179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9b434-ae4b-401a-8574-7e91fd48318a None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:47.350362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 57/107] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:47.356577 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cd716-ef1f-4e1c-9ece-48b1401f6484 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:47.357136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cd716-ef1f-4e1c-9ece-48b1401f6484 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:47.357599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cd716-ef1f-4e1c-9ece-48b1401f6484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:47.366210 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 51/108] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:12:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:12:47.373778 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb9797-dac9-4b24-aab6-eecaed6492c0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:47.405638 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3d4cd716-ef1f-4e1c-9ece-48b1401f6484 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:47.411627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3d4cd716-ef1f-4e1c-9ece-48b1401f6484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:47.554513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 58/109] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:12:47.569759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb9797-dac9-4b24-aab6-eecaed6492c0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:47.622173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e0424e-d614-4076-ad41-f1a9ba9c77be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:47.623316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0424e-d614-4076-ad41-f1a9ba9c77be None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:47.623738 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0424e-d614-4076-ad41-f1a9ba9c77be None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:47.624149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e0424e-d614-4076-ad41-f1a9ba9c77be None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:47.625561 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0424e-d614-4076-ad41-f1a9ba9c77be None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:47.627822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e0424e-d614-4076-ad41-f1a9ba9c77be None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:47.629699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 52/110] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:47 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:12:47.642837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cd716-ef1f-4e1c-9ece-48b1401f6484 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:47.658596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652852e-3c60-4a1c-943f-0e6911b2086c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:47.659106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652852e-3c60-4a1c-943f-0e6911b2086c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:47.659514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c652852e-3c60-4a1c-943f-0e6911b2086c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:47.661002 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 53/111] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:47.677836 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d505ad-3de0-4a3c-a657-241e2a34e28d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:47.678348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d505ad-3de0-4a3c-a657-241e2a34e28d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:47.678797 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d505ad-3de0-4a3c-a657-241e2a34e28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:47.711265 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c2d505ad-3de0-4a3c-a657-241e2a34e28d None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:47.720784 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c2d505ad-3de0-4a3c-a657-241e2a34e28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:47.778598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:47.802613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:47.898237 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4cd716-ef1f-4e1c-9ece-48b1401f6484 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:47.912685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 54/112] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:12:47.914207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7d58d-b25f-47d4-8e21-3337a6dd3eec None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:47.914828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7d58d-b25f-47d4-8e21-3337a6dd3eec None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:47.915247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e7d58d-b25f-47d4-8e21-3337a6dd3eec None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:47.916868 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 55/113] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:47.922337 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ecdcd-5711-4182-b021-15611c590ae8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:47.922862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ecdcd-5711-4182-b021-15611c590ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:47.923283 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ecdcd-5711-4182-b021-15611c590ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:47.941909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-690ecdcd-5711-4182-b021-15611c590ae8 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:47.943399 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-690ecdcd-5711-4182-b021-15611c590ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:47.978177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae39fc-30f3-44d6-a507-e122133142c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:47.979870 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae39fc-30f3-44d6-a507-e122133142c0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:47.981135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae39fc-30f3-44d6-a507-e122133142c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:47.983422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccae39fc-30f3-44d6-a507-e122133142c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:47.984328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae39fc-30f3-44d6-a507-e122133142c0 None admin] RBAC: Adding query filter params (role.id=5a6ded16e6f54de78e356e7b984522fa, scope.project.id=7e93b768826c4821ad3dd853154e62b4, user.id=5878318c80a4470181cbe55d2c509015) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:47.984786 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae39fc-30f3-44d6-a507-e122133142c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:47.990686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccae39fc-30f3-44d6-a507-e122133142c0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:48.003065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 59/114] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:12:47 2022] GET /identity/v3/role_assignments?user.id=5878318c80a4470181cbe55d2c509015&scope.project.id=7e93b768826c4821ad3dd853154e62b4&role.id=5a6ded16e6f54de78e356e7b984522fa => generated 259 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:48.054190 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 56/115] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:12:48.107913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae39fc-30f3-44d6-a507-e122133142c0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:48.203886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971f040-631f-41bb-a072-7001caef68ff None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:48.204408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971f040-631f-41bb-a072-7001caef68ff None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:48.204858 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f971f040-631f-41bb-a072-7001caef68ff None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:48.206379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 57/116] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:12:48.216043 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fad68f-f590-4938-96b6-65180b766d3e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:48.216671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fad68f-f590-4938-96b6-65180b766d3e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:48.217137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fad68f-f590-4938-96b6-65180b766d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:48.247052 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f6fad68f-f590-4938-96b6-65180b766d3e None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:48.248675 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f6fad68f-f590-4938-96b6-65180b766d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:48.271533 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccae39fc-30f3-44d6-a507-e122133142c0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:48.294526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:48.309610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641c6f2-07a9-4af0-b6bc-f310b202334d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:48.310268 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641c6f2-07a9-4af0-b6bc-f310b202334d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:48.310555 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641c6f2-07a9-4af0-b6bc-f310b202334d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:48.310829 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a641c6f2-07a9-4af0-b6bc-f310b202334d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:48.326207 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a641c6f2-07a9-4af0-b6bc-f310b202334d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:48.331884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-a641c6f2-07a9-4af0-b6bc-f310b202334d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:12:48.331884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:48.331884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:48.331884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:48.331884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:48.331884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:48.331884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:48.331884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:48.331884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:48.331884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:48.331884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:48.331884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:48.331884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:48.331884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:48.331884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:48.331884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:48.332783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:48.332783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:48.332783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:48.332783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:48.332783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:48.332783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:48.332783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:48.332783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:48.332783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:48.332783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:48.332783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:48.332783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:48.332783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:48.332783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:48.332783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:48.332783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:48.333520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:48.333520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:48.333520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:48.333520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:48.333520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:48.333520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:48.333520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:48.333520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:48.333520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:48.333520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:48.333520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:12:48.333520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:12:48.336260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 60/117] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:12:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:48.347819 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641c6f2-07a9-4af0-b6bc-f310b202334d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:48.363068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 58/118] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:12:48.435200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fad68f-f590-4938-96b6-65180b766d3e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:48.480151 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:48.487894 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a641c6f2-07a9-4af0-b6bc-f310b202334d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:48.530100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299df84-761c-4df1-8436-3ad69b975871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:48.531124 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299df84-761c-4df1-8436-3ad69b975871 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:48.531645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299df84-761c-4df1-8436-3ad69b975871 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:48.532280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3299df84-761c-4df1-8436-3ad69b975871 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:48.533485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299df84-761c-4df1-8436-3ad69b975871 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:48.533972 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299df84-761c-4df1-8436-3ad69b975871 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:48.535764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3299df84-761c-4df1-8436-3ad69b975871 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:48.546149 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 61/119] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:12:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:12:48.552079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299df84-761c-4df1-8436-3ad69b975871 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:48.607440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:48.656463 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fad68f-f590-4938-96b6-65180b766d3e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:48.692648 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3299df84-761c-4df1-8436-3ad69b975871 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:48.732795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc38063-10c8-46db-bcce-a5be0468ea0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:48.733799 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc38063-10c8-46db-bcce-a5be0468ea0f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:48.734342 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc38063-10c8-46db-bcce-a5be0468ea0f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:48.734929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc38063-10c8-46db-bcce-a5be0468ea0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:48.737695 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb640d-cb72-4607-b569-dd7a12b59a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:48.738457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb640d-cb72-4607-b569-dd7a12b59a64 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:48.738887 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb640d-cb72-4607-b569-dd7a12b59a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:48.739306 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eb640d-cb72-4607-b569-dd7a12b59a64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:48.749809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc38063-10c8-46db-bcce-a5be0468ea0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:48.757453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-5dc38063-10c8-46db-bcce-a5be0468ea0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:12:48.757453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:48.757453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:48.757453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:48.757453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:48.757453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:48.757453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:48.757453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:48.757453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:48.757453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:48.757453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:48.757453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:48.757453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:48.757453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:48.757453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:48.757453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:48.758318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:48.758318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:48.758318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:48.758318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:48.758318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:48.758318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:48.758318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:48.758318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:48.758318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:48.758318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:48.758318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:48.758318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:48.758318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:48.758318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:48.758318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:48.758318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:48.759086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:48.759086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:48.759086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:48.759086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:48.759086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:48.759086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:48.759086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:48.759086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:48.759086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:48.759086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:48.759086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:12:48.759086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:12:48.761635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 62/120] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:12:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:48.762837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc38063-10c8-46db-bcce-a5be0468ea0f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:48.773810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eb640d-cb72-4607-b569-dd7a12b59a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:48.779197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-76eb640d-cb72-4607-b569-dd7a12b59a64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:12:48.779197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:48.779197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:48.779197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:48.779197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:48.779197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:48.779197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:48.779197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:48.779197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:48.779197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:48.779197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:48.779197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:48.779197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:48.779197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:48.779197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:48.779197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:48.780057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:48.780057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:48.780057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:48.780057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:48.780057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:48.780057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:48.780057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:48.780057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:48.780057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:48.780057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:48.780057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:48.780057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:48.780057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:48.780057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:48.780057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:48.780057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:48.780807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:48.780807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:48.780807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:48.780807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:48.780807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:48.780807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:48.780807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:48.780807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:48.780807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:48.780807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:48.780807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:12:48.780807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:12:48.780807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 59/121] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:12:48 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:12:48.781452 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb640d-cb72-4607-b569-dd7a12b59a64 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:48.890324 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc38063-10c8-46db-bcce-a5be0468ea0f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:48.893059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:48.896627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:48.920906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eb640d-cb72-4607-b569-dd7a12b59a64 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:48.934617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ac591-62e3-40da-8c4c-9578e790bd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:48.935811 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac591-62e3-40da-8c4c-9578e790bd17 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:48.936245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac591-62e3-40da-8c4c-9578e790bd17 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:48.936923 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac591-62e3-40da-8c4c-9578e790bd17 None admin] PATH_INFO: `/v3/users/793b87d316d544f9aecda76f798927dc` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:48.938247 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ac591-62e3-40da-8c4c-9578e790bd17 None admin] RBAC: Authorizing `identity:get_user(user_id=793b87d316d544f9aecda76f798927dc)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:48.940450 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ac591-62e3-40da-8c4c-9578e790bd17 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:48.942709 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 63/122] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:48 2022] GET /identity/v3/users/793b87d316d544f9aecda76f798927dc => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:48.943914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ac591-62e3-40da-8c4c-9578e790bd17 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:48.965738 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e93a50-d982-4f61-be41-37e1a8a42b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:48.966832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e93a50-d982-4f61-be41-37e1a8a42b8b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:48.967314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e93a50-d982-4f61-be41-37e1a8a42b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:48.967933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e93a50-d982-4f61-be41-37e1a8a42b8b None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:48.968861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e93a50-d982-4f61-be41-37e1a8a42b8b None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:48.969368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e93a50-d982-4f61-be41-37e1a8a42b8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:48.971106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e93a50-d982-4f61-be41-37e1a8a42b8b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:48.980366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 60/123] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:12:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:48.986348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e93a50-d982-4f61-be41-37e1a8a42b8b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:49.064818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ac591-62e3-40da-8c4c-9578e790bd17 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:49.106729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816bf07-5e15-41c2-892e-efdcb281960d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:49.107729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816bf07-5e15-41c2-892e-efdcb281960d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:49.108232 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816bf07-5e15-41c2-892e-efdcb281960d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:49.108367 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e93a50-d982-4f61-be41-37e1a8a42b8b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:49.108774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d816bf07-5e15-41c2-892e-efdcb281960d None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:49.109660 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816bf07-5e15-41c2-892e-efdcb281960d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:49.110286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816bf07-5e15-41c2-892e-efdcb281960d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:49.111991 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d816bf07-5e15-41c2-892e-efdcb281960d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:49.122969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 64/124] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:12:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:49.124459 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816bf07-5e15-41c2-892e-efdcb281960d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:49.149823 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14739707-d30e-4a54-ba87-f23e62599cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:49.150791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14739707-d30e-4a54-ba87-f23e62599cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:49.151076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14739707-d30e-4a54-ba87-f23e62599cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:49.151355 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14739707-d30e-4a54-ba87-f23e62599cc9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:49.162210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:49.166259 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14739707-d30e-4a54-ba87-f23e62599cc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:49.172697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-14739707-d30e-4a54-ba87-f23e62599cc9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:12:49.172697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:49.172697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:49.172697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:49.172697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:49.172697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:49.172697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:49.172697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:49.172697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:49.172697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:49.172697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:49.172697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:49.172697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:49.172697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:49.172697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:49.172697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:49.173758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:49.173758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:49.173758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:49.173758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:49.173758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:49.173758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:49.173758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:49.173758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:49.173758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:49.173758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:49.173758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:49.173758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:49.173758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:49.173758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:49.173758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:49.173758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:49.179908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:49.179908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:49.179908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:49.179908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:49.179908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:49.179908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:49.179908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:49.179908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:49.179908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:49.179908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:49.179908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:12:49.179908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:12:49.179908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 61/125] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:12:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:49.179908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14739707-d30e-4a54-ba87-f23e62599cc9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:49.184581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:49.261744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d816bf07-5e15-41c2-892e-efdcb281960d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:49.297477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bdb2c-653d-4b17-bddd-087e8af2c43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:49.298113 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bdb2c-653d-4b17-bddd-087e8af2c43f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:49.298437 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bdb2c-653d-4b17-bddd-087e8af2c43f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:49.298712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594bdb2c-653d-4b17-bddd-087e8af2c43f None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:49.299902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bdb2c-653d-4b17-bddd-087e8af2c43f None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:49.300862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14739707-d30e-4a54-ba87-f23e62599cc9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:49.301352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594bdb2c-653d-4b17-bddd-087e8af2c43f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:49.306468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 65/126] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:49 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:49.307311 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bdb2c-653d-4b17-bddd-087e8af2c43f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:49.342979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8fd0f-9e44-482f-8565-40b3b3261b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:49.343637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8fd0f-9e44-482f-8565-40b3b3261b00 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:49.343925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8fd0f-9e44-482f-8565-40b3b3261b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:49.344208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8fd0f-9e44-482f-8565-40b3b3261b00 None admin] PATH_INFO: `/v3/users/179374916edf4dd480273f6c8c748ff6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:49.346358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb8fd0f-9e44-482f-8565-40b3b3261b00 None admin] RBAC: Authorizing `identity:get_user(user_id=179374916edf4dd480273f6c8c748ff6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:49.347834 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb8fd0f-9e44-482f-8565-40b3b3261b00 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:49.349438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 62/127] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:49 2022] GET /identity/v3/users/179374916edf4dd480273f6c8c748ff6 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:49.350243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8fd0f-9e44-482f-8565-40b3b3261b00 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:49.422488 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:49.428059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594bdb2c-653d-4b17-bddd-087e8af2c43f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:49.456293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:49.468828 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf6b2e-6322-4578-a0e2-feb64c5e6d59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:49.469519 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf6b2e-6322-4578-a0e2-feb64c5e6d59 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:49.469808 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf6b2e-6322-4578-a0e2-feb64c5e6d59 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:49.470084 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf6b2e-6322-4578-a0e2-feb64c5e6d59 None admin] PATH_INFO: `/v3/users/179374916edf4dd480273f6c8c748ff6` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:49.471356 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf6b2e-6322-4578-a0e2-feb64c5e6d59 None admin] RBAC: Authorizing `identity:get_user(user_id=179374916edf4dd480273f6c8c748ff6)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:49.473293 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf6b2e-6322-4578-a0e2-feb64c5e6d59 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:49.480193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 66/128] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:49 2022] GET /identity/v3/users/179374916edf4dd480273f6c8c748ff6 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:49.480193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf6b2e-6322-4578-a0e2-feb64c5e6d59 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:49.489009 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8fd0f-9e44-482f-8565-40b3b3261b00 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:49.525022 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb9a49-543c-4777-8f55-8a9118b6539c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:49.525814 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb9a49-543c-4777-8f55-8a9118b6539c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:49.526091 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb9a49-543c-4777-8f55-8a9118b6539c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:49.526390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb9a49-543c-4777-8f55-8a9118b6539c None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:49.529303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb9a49-543c-4777-8f55-8a9118b6539c None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:49.529303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb9a49-543c-4777-8f55-8a9118b6539c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:49.529303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb9a49-543c-4777-8f55-8a9118b6539c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:49.539624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 63/129] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:12:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:49.540547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb9a49-543c-4777-8f55-8a9118b6539c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:49.604477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf6b2e-6322-4578-a0e2-feb64c5e6d59 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:49.639730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5317bad1-f200-4758-8475-8bb3cf5fbb8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:49.640490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317bad1-f200-4758-8475-8bb3cf5fbb8f None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:49.640882 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317bad1-f200-4758-8475-8bb3cf5fbb8f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:49.641156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5317bad1-f200-4758-8475-8bb3cf5fbb8f None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4/users/793b87d316d544f9aecda76f798927dc/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:49.643271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317bad1-f200-4758-8475-8bb3cf5fbb8f None admin] RBAC: Authorizing `identity:create_grant(project_id=7e93b768826c4821ad3dd853154e62b4, user_id=793b87d316d544f9aecda76f798927dc, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:49.644826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5317bad1-f200-4758-8475-8bb3cf5fbb8f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:49.651467 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb9a49-543c-4777-8f55-8a9118b6539c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:49.666399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 67/130] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:12:49 2022] PUT /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4/users/793b87d316d544f9aecda76f798927dc/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:12:49.666399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1fd6e-e4dd-4371-9dec-34637091e632 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:49.666399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1fd6e-e4dd-4371-9dec-34637091e632 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:49.666399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1fd6e-e4dd-4371-9dec-34637091e632 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:49.666399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 68/131] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:49.666399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1fd6e-e4dd-4371-9dec-34637091e632 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:49.705149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0bba0-0442-448e-9ebc-039cf76ff3ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:49.705788 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0bba0-0442-448e-9ebc-039cf76ff3ab None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:49.706067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0bba0-0442-448e-9ebc-039cf76ff3ab None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:49.706373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0bba0-0442-448e-9ebc-039cf76ff3ab None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:49.707690 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0bba0-0442-448e-9ebc-039cf76ff3ab None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:49.709161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0bba0-0442-448e-9ebc-039cf76ff3ab None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:49.711672 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 64/132] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:49 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:49.712476 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0bba0-0442-448e-9ebc-039cf76ff3ab None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:49.715339 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:49.728084 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:49.829160 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0bba0-0442-448e-9ebc-039cf76ff3ab None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:49.849943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1fd6e-e4dd-4371-9dec-34637091e632 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:49.879064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfe3c2-dcf0-439c-8060-397b9e22cf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:49.879705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfe3c2-dcf0-439c-8060-397b9e22cf17 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:49.879984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfe3c2-dcf0-439c-8060-397b9e22cf17 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:49.880254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfe3c2-dcf0-439c-8060-397b9e22cf17 None admin] PATH_INFO: `/v3/users/179374916edf4dd480273f6c8c748ff6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:49.882477 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfe3c2-dcf0-439c-8060-397b9e22cf17 None admin] RBAC: Authorizing `identity:get_user(user_id=179374916edf4dd480273f6c8c748ff6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:49.883932 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfe3c2-dcf0-439c-8060-397b9e22cf17 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:49.889963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 65/133] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:49 2022] GET /identity/v3/users/179374916edf4dd480273f6c8c748ff6 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:49.889963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a949d-76ed-4812-a698-4caf28053e16 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:49.889963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a949d-76ed-4812-a698-4caf28053e16 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:49.889963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9a949d-76ed-4812-a698-4caf28053e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:49.905704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380968f-3efc-41c3-88d6-04f1264143b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:49.906452 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380968f-3efc-41c3-88d6-04f1264143b8 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:49.906648 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380968f-3efc-41c3-88d6-04f1264143b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:49.906913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380968f-3efc-41c3-88d6-04f1264143b8 None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:49.907969 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-df9a949d-76ed-4812-a698-4caf28053e16 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:49.908079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380968f-3efc-41c3-88d6-04f1264143b8 None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:49.909332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-df9a949d-76ed-4812-a698-4caf28053e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:49.909454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380968f-3efc-41c3-88d6-04f1264143b8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:49.911109 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 69/134] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:49 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:49.912213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380968f-3efc-41c3-88d6-04f1264143b8 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:49.999869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:50.010585 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 66/135] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:50.011693 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a949d-76ed-4812-a698-4caf28053e16 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:50.037508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:50.050118 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380968f-3efc-41c3-88d6-04f1264143b8 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:50.115860 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b03fd-c36b-4ae5-a44b-84ef7c1432b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:50.117064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b03fd-c36b-4ae5-a44b-84ef7c1432b4 None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:50.117346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b03fd-c36b-4ae5-a44b-84ef7c1432b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:50.117612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00b03fd-c36b-4ae5-a44b-84ef7c1432b4 None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4/users/179374916edf4dd480273f6c8c748ff6/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:50.119982 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b03fd-c36b-4ae5-a44b-84ef7c1432b4 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e93b768826c4821ad3dd853154e62b4, user_id=179374916edf4dd480273f6c8c748ff6, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:50.121442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00b03fd-c36b-4ae5-a44b-84ef7c1432b4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:50.146949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 70/136] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:12:49 2022] PUT /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4/users/179374916edf4dd480273f6c8c748ff6/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:12:50.147669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b03fd-c36b-4ae5-a44b-84ef7c1432b4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:50.201589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9a949d-76ed-4812-a698-4caf28053e16 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:50.269684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e0e608-8bf6-4cf7-a00d-e95a9d9ed09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:50.270375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0e608-8bf6-4cf7-a00d-e95a9d9ed09c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:50.270656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0e608-8bf6-4cf7-a00d-e95a9d9ed09c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:50.270932 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e0e608-8bf6-4cf7-a00d-e95a9d9ed09c None admin] PATH_INFO: `/v3/projects/39c9b647649b472bb19adde410a53a68` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:50.272070 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0e608-8bf6-4cf7-a00d-e95a9d9ed09c None admin] RBAC: Authorizing `identity:get_project(project_id=39c9b647649b472bb19adde410a53a68)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:50.274076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e0e608-8bf6-4cf7-a00d-e95a9d9ed09c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:50.276380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 67/137] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:50 2022] GET /identity/v3/projects/39c9b647649b472bb19adde410a53a68 => generated 312 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:50.277474 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acac044-a4c9-4347-bab4-5da3dbe609ba None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:50.277757 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acac044-a4c9-4347-bab4-5da3dbe609ba None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:50.278029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acac044-a4c9-4347-bab4-5da3dbe609ba None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:50.279562 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 68/138] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:50.284981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253fb98-2f89-450f-8667-a6df2189a400 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:50.285267 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253fb98-2f89-450f-8667-a6df2189a400 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:50.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2253fb98-2f89-450f-8667-a6df2189a400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:50.305896 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2253fb98-2f89-450f-8667-a6df2189a400 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:50.307135 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2253fb98-2f89-450f-8667-a6df2189a400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:50.349453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00b03fd-c36b-4ae5-a44b-84ef7c1432b4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:50.395309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:50.405655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 69/139] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:12:50.406619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253fb98-2f89-450f-8667-a6df2189a400 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:50.417543 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67369979-7df8-44d7-b4d7-7c2a522c574e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:50.418889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67369979-7df8-44d7-b4d7-7c2a522c574e None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:50.419385 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67369979-7df8-44d7-b4d7-7c2a522c574e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:50.419868 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67369979-7df8-44d7-b4d7-7c2a522c574e None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4/users/179374916edf4dd480273f6c8c748ff6/roles/d123411b649d4acb8648d0c0b80aae6d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:50.422435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67369979-7df8-44d7-b4d7-7c2a522c574e None admin] RBAC: Authorizing `identity:create_grant(project_id=7e93b768826c4821ad3dd853154e62b4, user_id=179374916edf4dd480273f6c8c748ff6, role_id=d123411b649d4acb8648d0c0b80aae6d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:50.424092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67369979-7df8-44d7-b4d7-7c2a522c574e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:50.446238 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 71/140] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:12:50 2022] PUT /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4/users/179374916edf4dd480273f6c8c748ff6/roles/d123411b649d4acb8648d0c0b80aae6d => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:12:50.447380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67369979-7df8-44d7-b4d7-7c2a522c574e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:50.474521 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:50.594449 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2253fb98-2f89-450f-8667-a6df2189a400 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:50.602803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67369979-7df8-44d7-b4d7-7c2a522c574e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:50.668387 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3ba32-4a57-4554-91e9-c2e3b6032680 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:50.669864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3ba32-4a57-4554-91e9-c2e3b6032680 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:50.671654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3ba32-4a57-4554-91e9-c2e3b6032680 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:50.672483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd79a53-0134-483c-86b5-8bb6f88329e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:50.673030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb3ba32-4a57-4554-91e9-c2e3b6032680 None admin] PATH_INFO: `/v3/projects/39c9b647649b472bb19adde410a53a68/users/179374916edf4dd480273f6c8c748ff6/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:50.674843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79a53-0134-483c-86b5-8bb6f88329e4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:50.675853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79a53-0134-483c-86b5-8bb6f88329e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:50.676315 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd79a53-0134-483c-86b5-8bb6f88329e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:50.677227 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3ba32-4a57-4554-91e9-c2e3b6032680 None admin] RBAC: Authorizing `identity:create_grant(project_id=39c9b647649b472bb19adde410a53a68, user_id=179374916edf4dd480273f6c8c748ff6, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:50.679653 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb3ba32-4a57-4554-91e9-c2e3b6032680 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:50.680487 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd79a53-0134-483c-86b5-8bb6f88329e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:50.682106 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd79a53-0134-483c-86b5-8bb6f88329e4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:50.684947 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 72/141] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:12:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:50.696068 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd79a53-0134-483c-86b5-8bb6f88329e4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:50.709580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 70/142] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:12:50 2022] PUT /identity/v3/projects/39c9b647649b472bb19adde410a53a68/users/179374916edf4dd480273f6c8c748ff6/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:12:50.822718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:50.842662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd79a53-0134-483c-86b5-8bb6f88329e4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:50.892143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e655d0-fdd0-4580-8e6d-d66796d20bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:50.892758 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e655d0-fdd0-4580-8e6d-d66796d20bcb None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:50.893034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e655d0-fdd0-4580-8e6d-d66796d20bcb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:50.893303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e655d0-fdd0-4580-8e6d-d66796d20bcb None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:50.893948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e655d0-fdd0-4580-8e6d-d66796d20bcb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:50.895430 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e655d0-fdd0-4580-8e6d-d66796d20bcb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:50.920356 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 73/143] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:12:50 2022] POST /identity/v3/users => generated 283 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:12:50.938118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3ba32-4a57-4554-91e9-c2e3b6032680 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:51.017490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:51.065568 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb3ba32-4a57-4554-91e9-c2e3b6032680 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:51.103370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a23988-78b0-49da-a171-740ec55a6daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:51.104032 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a23988-78b0-49da-a171-740ec55a6daa None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:51.104320 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a23988-78b0-49da-a171-740ec55a6daa None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:51.104591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a23988-78b0-49da-a171-740ec55a6daa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:51.117261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a23988-78b0-49da-a171-740ec55a6daa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:51.122727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-97a23988-78b0-49da-a171-740ec55a6daa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:12:51.122727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:51.122727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:51.122727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:51.122727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:51.122727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:51.122727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:51.122727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:51.122727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:51.122727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:51.122727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:51.122727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:51.122727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:51.122727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:51.122727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:51.122727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:51.123597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:51.123597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:51.123597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:51.123597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:51.123597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:51.123597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:51.123597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:51.123597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:51.123597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:51.123597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:51.123597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:51.123597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:51.123597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:51.123597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:51.123597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:51.123597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:51.124366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:51.124366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:51.124366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:51.124366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:51.124366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:51.124366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:51.124366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:51.124366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:51.124366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:51.124366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:51.124366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:12:51.124366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:12:51.124366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 71/144] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:12:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:51.135410 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e655d0-fdd0-4580-8e6d-d66796d20bcb None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:51.259353 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e655d0-fdd0-4580-8e6d-d66796d20bcb None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:51.295773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd228f7-1db3-4741-ab7d-39cf1a42e797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:51.296387 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd228f7-1db3-4741-ab7d-39cf1a42e797 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:51.296659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd228f7-1db3-4741-ab7d-39cf1a42e797 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:51.296929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd228f7-1db3-4741-ab7d-39cf1a42e797 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:51.297596 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd228f7-1db3-4741-ab7d-39cf1a42e797 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:51.297892 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd228f7-1db3-4741-ab7d-39cf1a42e797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:51.299302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd228f7-1db3-4741-ab7d-39cf1a42e797 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:51.307853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 74/145] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:12:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:51.313703 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a23988-78b0-49da-a171-740ec55a6daa None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:51.457155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a23988-78b0-49da-a171-740ec55a6daa None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:51.489674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:51.518444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b8840-9bca-497c-9361-e1b9600131e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:51.519439 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b8840-9bca-497c-9361-e1b9600131e1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:51.519957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b8840-9bca-497c-9361-e1b9600131e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:51.520454 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b8840-9bca-497c-9361-e1b9600131e1 None admin] PATH_INFO: `/v3/users/5878318c80a4470181cbe55d2c509015` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:51.521931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b8840-9bca-497c-9361-e1b9600131e1 None admin] RBAC: Authorizing `identity:get_user(user_id=5878318c80a4470181cbe55d2c509015)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:51.533160 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b8840-9bca-497c-9361-e1b9600131e1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:51.535262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 72/146] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:51 2022] GET /identity/v3/users/5878318c80a4470181cbe55d2c509015 => generated 289 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:51.542858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd228f7-1db3-4741-ab7d-39cf1a42e797 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:51.545765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:51.690817 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd228f7-1db3-4741-ab7d-39cf1a42e797 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:51.729715 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf8557d-5cf6-4a5e-a421-6df7255707c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:51.730386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8557d-5cf6-4a5e-a421-6df7255707c9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:51.730669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8557d-5cf6-4a5e-a421-6df7255707c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:51.730933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf8557d-5cf6-4a5e-a421-6df7255707c9 None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:51.732055 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8557d-5cf6-4a5e-a421-6df7255707c9 None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:51.733490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf8557d-5cf6-4a5e-a421-6df7255707c9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:51.735156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 75/147] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:51 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:51.740954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b8840-9bca-497c-9361-e1b9600131e1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:51.883102 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b8840-9bca-497c-9361-e1b9600131e1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:51.925922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d0c0cf-2166-45d6-a302-f3cc9c5d651d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:51.927029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0c0cf-2166-45d6-a302-f3cc9c5d651d None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:51.927521 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0c0cf-2166-45d6-a302-f3cc9c5d651d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:51.927981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d0c0cf-2166-45d6-a302-f3cc9c5d651d None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4/users/5878318c80a4470181cbe55d2c509015/roles/5a6ded16e6f54de78e356e7b984522fa` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:51.930906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0c0cf-2166-45d6-a302-f3cc9c5d651d None admin] RBAC: Authorizing `identity:create_grant(project_id=7e93b768826c4821ad3dd853154e62b4, user_id=5878318c80a4470181cbe55d2c509015, role_id=5a6ded16e6f54de78e356e7b984522fa)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:51.932570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d0c0cf-2166-45d6-a302-f3cc9c5d651d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:51.953790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 73/148] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:12:51 2022] PUT /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4/users/5878318c80a4470181cbe55d2c509015/roles/5a6ded16e6f54de78e356e7b984522fa => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:12:52.332549 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cdc98-1284-42df-908f-5d116d9d931f None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:52.333579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cdc98-1284-42df-908f-5d116d9d931f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:52.334073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218cdc98-1284-42df-908f-5d116d9d931f None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:52.336066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 76/149] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:52.341567 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd8dc3-c0b4-4552-9e09-95afb63ed389 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:52.342114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd8dc3-c0b4-4552-9e09-95afb63ed389 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:52.342593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebd8dc3-c0b4-4552-9e09-95afb63ed389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:52.368067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-eebd8dc3-c0b4-4552-9e09-95afb63ed389 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:52.369668 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-eebd8dc3-c0b4-4552-9e09-95afb63ed389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:52.423258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:52.457636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 74/150] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:12:52.660260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579bd87-0281-40ff-a152-955736698b42 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:52.660895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579bd87-0281-40ff-a152-955736698b42 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:52.661383 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d579bd87-0281-40ff-a152-955736698b42 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:52.662980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 77/151] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:52.669260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21b7e8-28f0-4b18-9d1c-3785bf086d9b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:52.669787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21b7e8-28f0-4b18-9d1c-3785bf086d9b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:52.670482 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e21b7e8-28f0-4b18-9d1c-3785bf086d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:52.696485 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8e21b7e8-28f0-4b18-9d1c-3785bf086d9b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:52.698134 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8e21b7e8-28f0-4b18-9d1c-3785bf086d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:52.774736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 75/152] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:52.960748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579bd87-0281-40ff-a152-955736698b42 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:53.041228 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:53.077272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d579bd87-0281-40ff-a152-955736698b42 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:53.117558 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64caf49f-296b-4c5c-8fab-e48bd5f3f05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:53.118569 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64caf49f-296b-4c5c-8fab-e48bd5f3f05d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:53.119064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64caf49f-296b-4c5c-8fab-e48bd5f3f05d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:53.119522 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64caf49f-296b-4c5c-8fab-e48bd5f3f05d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:53.133089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64caf49f-296b-4c5c-8fab-e48bd5f3f05d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:53.138960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-64caf49f-296b-4c5c-8fab-e48bd5f3f05d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:12:53.138960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:53.138960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:53.138960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:53.138960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:53.138960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:53.138960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:53.138960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:53.138960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:53.138960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:53.138960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:53.138960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:53.138960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:53.138960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:53.138960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:53.138960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:53.139865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:53.139865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:53.139865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:53.139865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:53.139865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:53.139865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:53.139865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:53.139865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:53.139865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:53.139865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:53.139865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:53.139865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:53.139865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:53.139865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:53.139865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:53.139865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:53.140668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:53.140668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:53.140668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:53.140668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:53.140668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:53.140668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:53.140668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:53.140668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:53.140668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:53.140668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:53.140668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:12:53.140668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:12:53.143075 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 78/153] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:12:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:53.149344 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21b7e8-28f0-4b18-9d1c-3785bf086d9b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:53.154152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64caf49f-296b-4c5c-8fab-e48bd5f3f05d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:53.306518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e21b7e8-28f0-4b18-9d1c-3785bf086d9b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:53.321173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:53.346454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64caf49f-296b-4c5c-8fab-e48bd5f3f05d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:53.380904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51b0a0-e4e9-4edb-ba76-de35efca0374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:53.383191 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51b0a0-e4e9-4edb-ba76-de35efca0374 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:53.383681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51b0a0-e4e9-4edb-ba76-de35efca0374 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:53.384152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c51b0a0-e4e9-4edb-ba76-de35efca0374 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:53.386059 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51b0a0-e4e9-4edb-ba76-de35efca0374 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:53.389956 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51b0a0-e4e9-4edb-ba76-de35efca0374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:53.391583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c51b0a0-e4e9-4edb-ba76-de35efca0374 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:53.410066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 76/154] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:12:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:53.414974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:53.418717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51b0a0-e4e9-4edb-ba76-de35efca0374 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:53.431508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e05ca-f6f9-42ca-80c1-35b130ba8a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:53.432154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e05ca-f6f9-42ca-80c1-35b130ba8a3e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:53.432431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e05ca-f6f9-42ca-80c1-35b130ba8a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:53.432696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e05ca-f6f9-42ca-80c1-35b130ba8a3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:53.474917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e05ca-f6f9-42ca-80c1-35b130ba8a3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:53.480855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-f51e05ca-f6f9-42ca-80c1-35b130ba8a3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:12:53.480855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:53.480855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:53.480855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:53.480855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:53.480855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:53.480855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:53.480855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:53.480855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:53.480855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:53.480855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:53.480855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:53.480855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:53.480855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:53.480855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:53.480855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:53.482391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:53.482391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:53.482391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:53.482391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:53.482391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:53.482391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:53.482391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:53.482391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:53.482391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:53.482391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:53.482391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:53.482391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:53.482391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:53.482391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:53.482391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:53.482391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:53.484074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:53.484074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:53.484074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:53.484074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:53.484074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:53.484074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:53.484074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:53.484074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:53.484074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:53.484074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:53.484074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:12:53.484074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:12:53.484074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 79/155] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:12:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:53.495283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e05ca-f6f9-42ca-80c1-35b130ba8a3e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:53.607538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c51b0a0-e4e9-4edb-ba76-de35efca0374 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:53.646666 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e05ca-f6f9-42ca-80c1-35b130ba8a3e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:53.671057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc0f76-f89f-4f2e-9e2c-31d69ffac429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:53.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc0f76-f89f-4f2e-9e2c-31d69ffac429 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:53.672625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc0f76-f89f-4f2e-9e2c-31d69ffac429 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:53.673197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc0f76-f89f-4f2e-9e2c-31d69ffac429 None admin] PATH_INFO: `/v3/users/793b87d316d544f9aecda76f798927dc` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:53.674531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc0f76-f89f-4f2e-9e2c-31d69ffac429 None admin] RBAC: Authorizing `identity:get_user(user_id=793b87d316d544f9aecda76f798927dc)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:53.676382 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cc0f76-f89f-4f2e-9e2c-31d69ffac429 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:53.678285 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 77/156] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:53 2022] GET /identity/v3/users/793b87d316d544f9aecda76f798927dc => generated 247 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:12:53.684419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc0f76-f89f-4f2e-9e2c-31d69ffac429 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:53.687858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596257d8-fb3e-42dc-904d-ad23bb76419f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:53.688490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596257d8-fb3e-42dc-904d-ad23bb76419f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:53.688762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596257d8-fb3e-42dc-904d-ad23bb76419f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:53.689071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596257d8-fb3e-42dc-904d-ad23bb76419f None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:53.689704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596257d8-fb3e-42dc-904d-ad23bb76419f None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:53.690001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596257d8-fb3e-42dc-904d-ad23bb76419f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:53.691466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596257d8-fb3e-42dc-904d-ad23bb76419f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:53.701032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 80/157] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:12:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:53.708330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596257d8-fb3e-42dc-904d-ad23bb76419f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:53.749140 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:53.814732 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:53.897359 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cc0f76-f89f-4f2e-9e2c-31d69ffac429 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:53.914597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596257d8-fb3e-42dc-904d-ad23bb76419f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:53.976106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89087d-6fd4-4ad6-bb13-af993858fe41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:53.977314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89087d-6fd4-4ad6-bb13-af993858fe41 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:53.978603 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89087d-6fd4-4ad6-bb13-af993858fe41 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:53.979786 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a89087d-6fd4-4ad6-bb13-af993858fe41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:53.992422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997759fd-aae5-45cb-8cc3-131e2831cfaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:53.996057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997759fd-aae5-45cb-8cc3-131e2831cfaa None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:53.996568 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997759fd-aae5-45cb-8cc3-131e2831cfaa None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:53.997047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997759fd-aae5-45cb-8cc3-131e2831cfaa None admin] PATH_INFO: `/v3/users/179374916edf4dd480273f6c8c748ff6` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:53.998500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997759fd-aae5-45cb-8cc3-131e2831cfaa None admin] RBAC: Authorizing `identity:get_user(user_id=179374916edf4dd480273f6c8c748ff6)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:54.002207 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997759fd-aae5-45cb-8cc3-131e2831cfaa None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:54.004102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 81/158] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:53 2022] GET /identity/v3/users/179374916edf4dd480273f6c8c748ff6 => generated 275 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:12:54.011026 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997759fd-aae5-45cb-8cc3-131e2831cfaa None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:54.135397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89087d-6fd4-4ad6-bb13-af993858fe41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:54.137385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a89087d-6fd4-4ad6-bb13-af993858fe41 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:54.204420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:54.222294 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997759fd-aae5-45cb-8cc3-131e2831cfaa None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:54.252194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:54.283140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b113883-6a88-49b0-bb51-f72e4c313988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:54.283789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b113883-6a88-49b0-bb51-f72e4c313988 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:54.284073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b113883-6a88-49b0-bb51-f72e4c313988 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:54.284349 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b113883-6a88-49b0-bb51-f72e4c313988 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:54.336790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 78/159] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:12:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 653 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 15:12:54.342832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89087d-6fd4-4ad6-bb13-af993858fe41 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:54.418252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b113883-6a88-49b0-bb51-f72e4c313988 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:54.418252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b113883-6a88-49b0-bb51-f72e4c313988 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:54.517959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a89087d-6fd4-4ad6-bb13-af993858fe41 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:54.556175 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 82/160] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:12:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:12:54.557460 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745d715-95fb-4877-94d1-071dda600864 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:54.557748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745d715-95fb-4877-94d1-071dda600864 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:54.558017 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7745d715-95fb-4877-94d1-071dda600864 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:54.568714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 83/161] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:54 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:54.568714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745d715-95fb-4877-94d1-071dda600864 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:54.584790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4579c569-bcda-4ad8-9eaa-bf0d2257d309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:54.586066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579c569-bcda-4ad8-9eaa-bf0d2257d309 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:54.586416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579c569-bcda-4ad8-9eaa-bf0d2257d309 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:54.586770 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4579c569-bcda-4ad8-9eaa-bf0d2257d309 None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:54.588100 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579c569-bcda-4ad8-9eaa-bf0d2257d309 None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:54.589868 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4579c569-bcda-4ad8-9eaa-bf0d2257d309 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:54.592132 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 79/162] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:54 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:54.593362 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41652e89-38e7-45d5-9efa-e257d245fe9a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:54.593672 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41652e89-38e7-45d5-9efa-e257d245fe9a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:54.593942 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41652e89-38e7-45d5-9efa-e257d245fe9a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:54.595409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 80/163] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:54.596302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39af93-f909-4a5c-babc-62541032ec91 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:54.596582 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39af93-f909-4a5c-babc-62541032ec91 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:54.596852 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39af93-f909-4a5c-babc-62541032ec91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:54.608557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:54.609611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:54.624085 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9f39af93-f909-4a5c-babc-62541032ec91 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:54.625286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9f39af93-f909-4a5c-babc-62541032ec91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:54.699720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 81/164] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:54.700644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39af93-f909-4a5c-babc-62541032ec91 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:54.758908 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7745d715-95fb-4877-94d1-071dda600864 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:54.831801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1f86d-9b28-47c1-9bb4-0c6c92a5e14b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:54.832378 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1f86d-9b28-47c1-9bb4-0c6c92a5e14b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:54.832653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1f86d-9b28-47c1-9bb4-0c6c92a5e14b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:54.832918 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1f86d-9b28-47c1-9bb4-0c6c92a5e14b None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:54.834048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c1f86d-9b28-47c1-9bb4-0c6c92a5e14b None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:54.836067 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c1f86d-9b28-47c1-9bb4-0c6c92a5e14b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:54.837700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 84/165] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:54 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:54.838839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e759b-5a20-45b6-9ca6-d44095dbd13f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:54.839116 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e759b-5a20-45b6-9ca6-d44095dbd13f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:54.839380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8e759b-5a20-45b6-9ca6-d44095dbd13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:54.857028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39af93-f909-4a5c-babc-62541032ec91 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:54.869704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7e8e759b-5a20-45b6-9ca6-d44095dbd13f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:54.870948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7e8e759b-5a20-45b6-9ca6-d44095dbd13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:54.901836 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f7ab4-c463-445b-aee2-98e42a5c9761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:54.902630 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f7ab4-c463-445b-aee2-98e42a5c9761 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:54.902964 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f7ab4-c463-445b-aee2-98e42a5c9761 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:54.903268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5f7ab4-c463-445b-aee2-98e42a5c9761 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:54.904425 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f7ab4-c463-445b-aee2-98e42a5c9761 None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=7e93b768826c4821ad3dd853154e62b4, user.id=793b87d316d544f9aecda76f798927dc) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:54.904810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f7ab4-c463-445b-aee2-98e42a5c9761 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:54.909777 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5f7ab4-c463-445b-aee2-98e42a5c9761 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:54.919463 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:54.927714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 82/166] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:12:54 2022] GET /identity/v3/role_assignments?user.id=793b87d316d544f9aecda76f798927dc&scope.project.id=7e93b768826c4821ad3dd853154e62b4&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:54.928871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f7ab4-c463-445b-aee2-98e42a5c9761 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:55.033051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 85/167] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:55.115791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5f7ab4-c463-445b-aee2-98e42a5c9761 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:55.159703 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac515817-b074-48c9-a504-11d04a07d3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:55.160354 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac515817-b074-48c9-a504-11d04a07d3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:55.160634 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac515817-b074-48c9-a504-11d04a07d3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:55.160922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac515817-b074-48c9-a504-11d04a07d3b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:55.161680 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac515817-b074-48c9-a504-11d04a07d3b0 None admin] RBAC: Adding query filter params (role.id=8d794ef8abb94809bd988c5db32c0037, scope.project.id=7e93b768826c4821ad3dd853154e62b4, user.id=179374916edf4dd480273f6c8c748ff6) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:55.161982 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac515817-b074-48c9-a504-11d04a07d3b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:55.163511 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac515817-b074-48c9-a504-11d04a07d3b0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:55.175485 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 83/168] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:12:54 2022] GET /identity/v3/role_assignments?user.id=179374916edf4dd480273f6c8c748ff6&scope.project.id=7e93b768826c4821ad3dd853154e62b4&role.id=8d794ef8abb94809bd988c5db32c0037 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:55.425101 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e759b-5a20-45b6-9ca6-d44095dbd13f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:55.451001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:55.570213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8e759b-5a20-45b6-9ca6-d44095dbd13f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:55.618326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b43ea7-f3f3-4e97-a832-7011a4e21a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:55.618972 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b43ea7-f3f3-4e97-a832-7011a4e21a70 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:55.619249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b43ea7-f3f3-4e97-a832-7011a4e21a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:55.619520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b43ea7-f3f3-4e97-a832-7011a4e21a70 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:55.639228 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b43ea7-f3f3-4e97-a832-7011a4e21a70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:55.644283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-13b43ea7-f3f3-4e97-a832-7011a4e21a70 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:12:55.644283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:55.644283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:55.644283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:55.644283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:55.644283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:55.644283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:55.644283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:55.644283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:55.644283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:55.644283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:55.644283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:55.644283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:55.644283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:55.644283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:55.644283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:55.645427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:55.645427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:55.645427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:55.645427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:55.645427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:55.645427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:55.645427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:55.645427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:55.645427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:55.645427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:55.645427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:55.645427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:55.645427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:55.645427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:55.645427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:55.645427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:55.646264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:55.646264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:55.646264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:55.646264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:55.646264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:55.646264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:55.646264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:55.646264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:55.646264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:55.646264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:55.646264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:12:55.646264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:12:55.646264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 86/169] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:12:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:55.651983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac515817-b074-48c9-a504-11d04a07d3b0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:55.742172 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:55.790824 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac515817-b074-48c9-a504-11d04a07d3b0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:55.820339 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c695-0773-410f-94a3-f7c015afafbf None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:55.821096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c695-0773-410f-94a3-f7c015afafbf None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:55.821747 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b659c695-0773-410f-94a3-f7c015afafbf None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:55.823511 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 87/170] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:55.824492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659c695-0773-410f-94a3-f7c015afafbf None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:55.855645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff67ba48-27d3-4192-84b1-b85a53000ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:55.856331 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67ba48-27d3-4192-84b1-b85a53000ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:55.856616 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67ba48-27d3-4192-84b1-b85a53000ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:55.856883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff67ba48-27d3-4192-84b1-b85a53000ec4 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:55.858787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67ba48-27d3-4192-84b1-b85a53000ec4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:55.858787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67ba48-27d3-4192-84b1-b85a53000ec4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:55.859216 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff67ba48-27d3-4192-84b1-b85a53000ec4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:55.875514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 84/171] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:12:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:55.877109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3e8be-7196-4df4-a81e-84c014690f17 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:55.878608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3e8be-7196-4df4-a81e-84c014690f17 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:55.879231 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb3e8be-7196-4df4-a81e-84c014690f17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:55.918410 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-efb3e8be-7196-4df4-a81e-84c014690f17 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:55.920503 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-efb3e8be-7196-4df4-a81e-84c014690f17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:56.019819 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:56.118934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b659c695-0773-410f-94a3-f7c015afafbf None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:56.120384 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 85/172] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:56.121358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3e8be-7196-4df4-a81e-84c014690f17 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:56.209154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a7e06-e3fc-494b-98fc-fbec6fcb6cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:56.210461 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a7e06-e3fc-494b-98fc-fbec6fcb6cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:56.210672 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a7e06-e3fc-494b-98fc-fbec6fcb6cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:56.211782 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a7e06-e3fc-494b-98fc-fbec6fcb6cc0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:56.224596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:56.229566 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2a7e06-e3fc-494b-98fc-fbec6fcb6cc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:56.237760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-7a2a7e06-e3fc-494b-98fc-fbec6fcb6cc0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:12:56.237760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:56.237760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:56.237760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:56.237760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:56.237760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:56.237760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:56.237760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:56.237760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:56.237760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:56.237760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:56.237760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:56.237760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:56.237760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:56.237760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:56.237760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:56.238664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:56.238664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:56.238664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:56.238664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:56.238664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:56.238664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:56.238664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:56.238664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:56.238664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:56.238664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:56.238664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:56.238664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:56.238664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:56.238664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:56.238664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:56.238664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:56.239526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:56.239526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:56.239526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:56.239526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:56.239526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:56.239526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:56.239526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:56.239526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:56.239526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:56.239526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:56.239526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:12:56.239526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:12:56.239526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 88/173] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:12:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 415 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:56.240176 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2964102-106f-4a60-bc79-e0c68fe39ebd None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:56.245876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2964102-106f-4a60-bc79-e0c68fe39ebd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:56.245876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2964102-106f-4a60-bc79-e0c68fe39ebd None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:56.245876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 89/174] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:56.245876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4618a-6040-4c29-aa82-ae03d67762e7 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:56.245876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4618a-6040-4c29-aa82-ae03d67762e7 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:56.245876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4618a-6040-4c29-aa82-ae03d67762e7 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:56.246327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 90/175] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:56.250274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4618a-6040-4c29-aa82-ae03d67762e7 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:56.294567 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb3e8be-7196-4df4-a81e-84c014690f17 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:56.357261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49739cdf-e779-43c3-97e0-d69e52ecc039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:56.357939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49739cdf-e779-43c3-97e0-d69e52ecc039 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:56.358095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49739cdf-e779-43c3-97e0-d69e52ecc039 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:56.358533 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49739cdf-e779-43c3-97e0-d69e52ecc039 None admin] PATH_INFO: `/v3/users/179374916edf4dd480273f6c8c748ff6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:56.359664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49739cdf-e779-43c3-97e0-d69e52ecc039 None admin] RBAC: Authorizing `identity:get_user(user_id=179374916edf4dd480273f6c8c748ff6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:56.361335 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49739cdf-e779-43c3-97e0-d69e52ecc039 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:56.362715 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 86/176] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:56 2022] GET /identity/v3/users/179374916edf4dd480273f6c8c748ff6 => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:56.369552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fc9c3-0b64-420e-b1b8-27f948591b0f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:56.369885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fc9c3-0b64-420e-b1b8-27f948591b0f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:56.370333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fc9c3-0b64-420e-b1b8-27f948591b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:56.383187 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4618a-6040-4c29-aa82-ae03d67762e7 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:56.394329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c13fc9c3-0b64-420e-b1b8-27f948591b0f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:56.395736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c13fc9c3-0b64-420e-b1b8-27f948591b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:56.425997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708f44e-68e8-49bb-93f1-5e92e23fd3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:56.426791 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708f44e-68e8-49bb-93f1-5e92e23fd3f1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:56.427203 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708f44e-68e8-49bb-93f1-5e92e23fd3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:56.427530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708f44e-68e8-49bb-93f1-5e92e23fd3f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:56.428457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708f44e-68e8-49bb-93f1-5e92e23fd3f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:56.429303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708f44e-68e8-49bb-93f1-5e92e23fd3f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:56.430931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708f44e-68e8-49bb-93f1-5e92e23fd3f1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:56.436921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:56.442261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 91/177] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:12:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:56.442426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126c200-ba61-4682-aa94-a0eeb19bc6db None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:56.443069 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126c200-ba61-4682-aa94-a0eeb19bc6db None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:56.443069 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c126c200-ba61-4682-aa94-a0eeb19bc6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:56.464148 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c126c200-ba61-4682-aa94-a0eeb19bc6db None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:56.465350 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c126c200-ba61-4682-aa94-a0eeb19bc6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:56.478358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 87/178] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:56.480132 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ef676-5671-4e6f-9da8-ebc03452bfd5 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:56.480789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ef676-5671-4e6f-9da8-ebc03452bfd5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:56.481062 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758ef676-5671-4e6f-9da8-ebc03452bfd5 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:56.482637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 88/179] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:12:56.483392 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ef676-5671-4e6f-9da8-ebc03452bfd5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:56.559266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:56.559862 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 92/180] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:56.561432 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126c200-ba61-4682-aa94-a0eeb19bc6db None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:56.629434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758ef676-5671-4e6f-9da8-ebc03452bfd5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:56.675713 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4187e47b-ea89-43b8-819d-6e717ec48b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:56.676865 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187e47b-ea89-43b8-819d-6e717ec48b98 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:56.677557 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187e47b-ea89-43b8-819d-6e717ec48b98 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:56.678006 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4187e47b-ea89-43b8-819d-6e717ec48b98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:56.683503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c126c200-ba61-4682-aa94-a0eeb19bc6db None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:56.715804 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:56.723476 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff5e21-81dd-4a87-ad04-7f88becab4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:56.724248 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff5e21-81dd-4a87-ad04-7f88becab4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:56.724617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff5e21-81dd-4a87-ad04-7f88becab4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:56.724958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff5e21-81dd-4a87-ad04-7f88becab4b9 None admin] PATH_INFO: `/v3/users/179374916edf4dd480273f6c8c748ff6` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:56.726348 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff5e21-81dd-4a87-ad04-7f88becab4b9 None admin] RBAC: Authorizing `identity:get_user(user_id=179374916edf4dd480273f6c8c748ff6)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:56.728012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff5e21-81dd-4a87-ad04-7f88becab4b9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:56.729748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 93/181] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:56 2022] GET /identity/v3/users/179374916edf4dd480273f6c8c748ff6 => generated 275 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:56.735147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff5e21-81dd-4a87-ad04-7f88becab4b9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:56.769552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187e47b-ea89-43b8-819d-6e717ec48b98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:56.771045 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4187e47b-ea89-43b8-819d-6e717ec48b98 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:56.829732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:56.847527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff5e21-81dd-4a87-ad04-7f88becab4b9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:56.878770 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 89/182] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:12:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:12:56.880115 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631c122-89a7-4889-a168-344ae9464588 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:56.880401 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631c122-89a7-4889-a168-344ae9464588 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:56.880672 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631c122-89a7-4889-a168-344ae9464588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:56.886404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86898233-41c5-4aee-a081-fae95bc44b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:56.887031 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86898233-41c5-4aee-a081-fae95bc44b82 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:56.887307 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86898233-41c5-4aee-a081-fae95bc44b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:56.887603 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86898233-41c5-4aee-a081-fae95bc44b82 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:56.898774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86898233-41c5-4aee-a081-fae95bc44b82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:56.899663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9631c122-89a7-4889-a168-344ae9464588 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:12:56.901134 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9631c122-89a7-4889-a168-344ae9464588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:12:56.905037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-86898233-41c5-4aee-a081-fae95bc44b82 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:12:56.905037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:56.905037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:56.905037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:56.905037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:56.905037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:56.905037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:56.905037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:56.905037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:56.905037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:56.905037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:56.905037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:56.905037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:56.905037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:56.905037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:56.905037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:56.905896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:56.905896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:56.905896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:56.905896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:56.905896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:56.905896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:56.905896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:56.905896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:56.905896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:56.905896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:56.905896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:56.905896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:56.905896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:56.905896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:56.905896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:56.905896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:56.908089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:56.908089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:56.908089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:56.908089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:56.908089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:56.908089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:56.908089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:56.908089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:56.908089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:56.908089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:56.908089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:12:56.908089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:12:56.908089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 94/183] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:12:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:56.908089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86898233-41c5-4aee-a081-fae95bc44b82 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:56.969853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:56.970587 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 90/184] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:12:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:12:56.971778 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631c122-89a7-4889-a168-344ae9464588 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:57.013509 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86898233-41c5-4aee-a081-fae95bc44b82 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:57.050129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bf1e8-093c-43c9-a0cc-9821575964a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:57.050764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bf1e8-093c-43c9-a0cc-9821575964a1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:57.051043 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bf1e8-093c-43c9-a0cc-9821575964a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:57.051312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328bf1e8-093c-43c9-a0cc-9821575964a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:57.066600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:57.088003 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631c122-89a7-4889-a168-344ae9464588 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:57.119651 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328bf1e8-093c-43c9-a0cc-9821575964a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:57.121129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328bf1e8-093c-43c9-a0cc-9821575964a1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:57.128517 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd688561-9913-40ac-8913-dab920e5a332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:57.129269 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd688561-9913-40ac-8913-dab920e5a332 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:57.129546 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd688561-9913-40ac-8913-dab920e5a332 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:57.129813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd688561-9913-40ac-8913-dab920e5a332 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:57.141442 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd688561-9913-40ac-8913-dab920e5a332 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:57.146721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-dd688561-9913-40ac-8913-dab920e5a332 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:12:57.146721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:57.146721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:57.146721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:57.146721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:57.146721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:57.146721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:57.146721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:57.146721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:57.146721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:57.146721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:57.146721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:57.146721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:57.146721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:57.146721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:57.146721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:57.147629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:57.147629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:57.147629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:57.147629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:57.147629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:57.147629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:57.147629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:57.147629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:57.147629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:57.147629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:57.147629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:57.147629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:57.147629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:57.147629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:57.147629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:57.147629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:57.151250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:57.151250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:57.151250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:57.151250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:57.151250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:57.151250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:57.151250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:57.151250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:57.151250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:57.151250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:57.151250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:12:57.151250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:12:57.151250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 91/185] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:12:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:57.151250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd688561-9913-40ac-8913-dab920e5a332 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:57.182899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:57.231186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 95/186] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:12:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:12:57.232418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bf1e8-093c-43c9-a0cc-9821575964a1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:57.266471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd688561-9913-40ac-8913-dab920e5a332 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:57.300943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6866f6-2186-48f7-abe2-d0b1c9f61825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:57.301572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6866f6-2186-48f7-abe2-d0b1c9f61825 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:57.301851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6866f6-2186-48f7-abe2-d0b1c9f61825 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:57.302123 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6866f6-2186-48f7-abe2-d0b1c9f61825 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:57.303270 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6866f6-2186-48f7-abe2-d0b1c9f61825 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:57.305255 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6866f6-2186-48f7-abe2-d0b1c9f61825 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:57.306860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 92/187] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:12:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:57.310521 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6866f6-2186-48f7-abe2-d0b1c9f61825 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:57.320545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:57.335126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328bf1e8-093c-43c9-a0cc-9821575964a1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:57.371174 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5b66f-a76f-4463-833e-580d6ed3ebc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:57.371788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5b66f-a76f-4463-833e-580d6ed3ebc0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:57.372081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5b66f-a76f-4463-833e-580d6ed3ebc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:57.372407 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b5b66f-a76f-4463-833e-580d6ed3ebc0 None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:57.373521 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5b66f-a76f-4463-833e-580d6ed3ebc0 None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:57.375033 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b5b66f-a76f-4463-833e-580d6ed3ebc0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:57.376942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 96/188] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:57 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:57.377993 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5b66f-a76f-4463-833e-580d6ed3ebc0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:57.406739 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:57.408676 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6866f6-2186-48f7-abe2-d0b1c9f61825 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:57.444468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34451ec-f61f-47e4-8fec-39ae27556d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:57.445070 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34451ec-f61f-47e4-8fec-39ae27556d83 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:57.445349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34451ec-f61f-47e4-8fec-39ae27556d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:57.445618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34451ec-f61f-47e4-8fec-39ae27556d83 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:57.446312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34451ec-f61f-47e4-8fec-39ae27556d83 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:57.446612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34451ec-f61f-47e4-8fec-39ae27556d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:57.447971 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34451ec-f61f-47e4-8fec-39ae27556d83 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:57.455868 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 93/189] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:12:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:57.456664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34451ec-f61f-47e4-8fec-39ae27556d83 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:57.483538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b5b66f-a76f-4463-833e-580d6ed3ebc0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:57.521131 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29644b8f-97f8-4dca-8ac2-9b527f66d33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:57.521730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29644b8f-97f8-4dca-8ac2-9b527f66d33f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:57.522004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29644b8f-97f8-4dca-8ac2-9b527f66d33f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:57.522365 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29644b8f-97f8-4dca-8ac2-9b527f66d33f None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:57.523025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29644b8f-97f8-4dca-8ac2-9b527f66d33f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:57.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29644b8f-97f8-4dca-8ac2-9b527f66d33f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:57.524679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29644b8f-97f8-4dca-8ac2-9b527f66d33f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:57.529772 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:57.534249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 97/190] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:12:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:57.539203 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29644b8f-97f8-4dca-8ac2-9b527f66d33f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:57.561146 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34451ec-f61f-47e4-8fec-39ae27556d83 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:57.594541 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9833ea96-46f1-4415-b787-78ed0f390aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:57.595168 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9833ea96-46f1-4415-b787-78ed0f390aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:57.595449 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9833ea96-46f1-4415-b787-78ed0f390aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:57.595721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9833ea96-46f1-4415-b787-78ed0f390aa1 None admin] PATH_INFO: `/v3/projects/39c9b647649b472bb19adde410a53a68` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:57.596871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9833ea96-46f1-4415-b787-78ed0f390aa1 None admin] RBAC: Authorizing `identity:get_project(project_id=39c9b647649b472bb19adde410a53a68)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:57.598340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9833ea96-46f1-4415-b787-78ed0f390aa1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:57.599975 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 94/191] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:57 2022] GET /identity/v3/projects/39c9b647649b472bb19adde410a53a68 => generated 312 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:57.601225 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9833ea96-46f1-4415-b787-78ed0f390aa1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:57.628578 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:57.641053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29644b8f-97f8-4dca-8ac2-9b527f66d33f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:57.674604 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0787d-509f-4443-b6a0-41e124879790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:57.675239 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0787d-509f-4443-b6a0-41e124879790 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:57.675516 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0787d-509f-4443-b6a0-41e124879790 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:57.675789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b0787d-509f-4443-b6a0-41e124879790 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:57.686824 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b0787d-509f-4443-b6a0-41e124879790 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:12:57.692619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-31b0787d-509f-4443-b6a0-41e124879790 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:12:57.692619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:12:57.692619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:12:57.692619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:12:57.692619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:12:57.692619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:12:57.692619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:12:57.692619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:12:57.692619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:12:57.692619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:12:57.692619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:12:57.692619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:12:57.692619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:12:57.692619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:12:57.692619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:12:57.692619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:12:57.693685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:12:57.693685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:12:57.693685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:12:57.693685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:12:57.693685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:12:57.693685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:12:57.693685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:12:57.693685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:12:57.693685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:12:57.693685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:12:57.693685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:12:57.693685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:12:57.693685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:12:57.693685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:12:57.693685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:12:57.693685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:12:57.694743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:12:57.694743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:12:57.694743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:12:57.694743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:12:57.694743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:12:57.694743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:12:57.694743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:12:57.694743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:12:57.694743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:12:57.694743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:12:57.694743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:12:57.694743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:12:57.694743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 98/192] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:12:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:12:57.694743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0787d-509f-4443-b6a0-41e124879790 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:57.702445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9833ea96-46f1-4415-b787-78ed0f390aa1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:57.737025 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe5762-9852-4da5-96bb-db7e70b13708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:57.737643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe5762-9852-4da5-96bb-db7e70b13708 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:57.737907 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe5762-9852-4da5-96bb-db7e70b13708 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:57.738178 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfe5762-9852-4da5-96bb-db7e70b13708 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:57.738847 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe5762-9852-4da5-96bb-db7e70b13708 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:57.740280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfe5762-9852-4da5-96bb-db7e70b13708 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:57.752443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:57.762498 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 95/193] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:12:57 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:12:57.776813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe5762-9852-4da5-96bb-db7e70b13708 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:57.810614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b0787d-509f-4443-b6a0-41e124879790 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:57.850087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b2b65-3e87-4f55-b694-8877216cbcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:57.851362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b2b65-3e87-4f55-b694-8877216cbcfa None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:57.851636 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b2b65-3e87-4f55-b694-8877216cbcfa None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:57.852261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b2b65-3e87-4f55-b694-8877216cbcfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:57.853013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b2b65-3e87-4f55-b694-8877216cbcfa None admin] RBAC: Adding query filter params (role.id=d123411b649d4acb8648d0c0b80aae6d, scope.project.id=7e93b768826c4821ad3dd853154e62b4, user.id=179374916edf4dd480273f6c8c748ff6) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:57.853318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b2b65-3e87-4f55-b694-8877216cbcfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:57.854822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9b2b65-3e87-4f55-b694-8877216cbcfa None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:57.861097 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 99/194] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:12:57 2022] GET /identity/v3/role_assignments?user.id=179374916edf4dd480273f6c8c748ff6&scope.project.id=7e93b768826c4821ad3dd853154e62b4&role.id=d123411b649d4acb8648d0c0b80aae6d => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:57.861934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b2b65-3e87-4f55-b694-8877216cbcfa None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:57.885267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:57.921920 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfe5762-9852-4da5-96bb-db7e70b13708 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:57.969899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976cee3-c0ae-4e58-8b09-4b5d367f5fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:57.970623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976cee3-c0ae-4e58-8b09-4b5d367f5fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:57.971188 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976cee3-c0ae-4e58-8b09-4b5d367f5fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:57.971188 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976cee3-c0ae-4e58-8b09-4b5d367f5fd0 None admin] PATH_INFO: `/v3/users/5878318c80a4470181cbe55d2c509015` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:57.972603 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976cee3-c0ae-4e58-8b09-4b5d367f5fd0 None admin] RBAC: Authorizing `identity:get_user(user_id=5878318c80a4470181cbe55d2c509015)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:57.974301 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d976cee3-c0ae-4e58-8b09-4b5d367f5fd0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:57.978060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 96/195] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:57 2022] GET /identity/v3/users/5878318c80a4470181cbe55d2c509015 => generated 289 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:57.978060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976cee3-c0ae-4e58-8b09-4b5d367f5fd0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:58.023777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9b2b65-3e87-4f55-b694-8877216cbcfa None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:58.068424 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:58.079198 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af324e92-d90f-4f8e-b306-e71d0576e96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:58.079843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af324e92-d90f-4f8e-b306-e71d0576e96e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:58.080116 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af324e92-d90f-4f8e-b306-e71d0576e96e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:58.080390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af324e92-d90f-4f8e-b306-e71d0576e96e None admin] PATH_INFO: `/v3/users/273750f0431f44e8b0f7fc158b0b2832` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:58.102623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af324e92-d90f-4f8e-b306-e71d0576e96e None admin] RBAC: Authorizing `identity:get_user(user_id=273750f0431f44e8b0f7fc158b0b2832)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:58.104279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af324e92-d90f-4f8e-b306-e71d0576e96e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:58.107448 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 100/196] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:57 2022] GET /identity/v3/users/273750f0431f44e8b0f7fc158b0b2832 => generated 283 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:58.108361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af324e92-d90f-4f8e-b306-e71d0576e96e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:58.137138 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d976cee3-c0ae-4e58-8b09-4b5d367f5fd0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:58.178998 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fabef2-29ed-4302-80ce-6dfe64f1dde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:58.179618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fabef2-29ed-4302-80ce-6dfe64f1dde2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:58.179893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fabef2-29ed-4302-80ce-6dfe64f1dde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:58.180161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fabef2-29ed-4302-80ce-6dfe64f1dde2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:58.180893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fabef2-29ed-4302-80ce-6dfe64f1dde2 None admin] RBAC: Adding query filter params (role.id=8d794ef8abb94809bd988c5db32c0037, scope.project.id=39c9b647649b472bb19adde410a53a68, user.id=179374916edf4dd480273f6c8c748ff6) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:58.181198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fabef2-29ed-4302-80ce-6dfe64f1dde2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:58.184397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fabef2-29ed-4302-80ce-6dfe64f1dde2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:58.191438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 97/197] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:12:57 2022] GET /identity/v3/role_assignments?user.id=179374916edf4dd480273f6c8c748ff6&scope.project.id=39c9b647649b472bb19adde410a53a68&role.id=8d794ef8abb94809bd988c5db32c0037 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:58.192259 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fabef2-29ed-4302-80ce-6dfe64f1dde2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:58.204999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:58.264640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af324e92-d90f-4f8e-b306-e71d0576e96e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:58.337835 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f612b64-ee4d-49f1-8045-2f2d518577d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:58.338487 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f612b64-ee4d-49f1-8045-2f2d518577d1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:58.338761 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f612b64-ee4d-49f1-8045-2f2d518577d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:58.339026 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f612b64-ee4d-49f1-8045-2f2d518577d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:58.339703 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f612b64-ee4d-49f1-8045-2f2d518577d1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:58.340002 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f612b64-ee4d-49f1-8045-2f2d518577d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:58.341429 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f612b64-ee4d-49f1-8045-2f2d518577d1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:58.355990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 101/198] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:12:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:58.357970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f612b64-ee4d-49f1-8045-2f2d518577d1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:58.387819 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fabef2-29ed-4302-80ce-6dfe64f1dde2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:58.433758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:58.446664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9931ad-4f93-4fa1-9474-1ae9cec6f54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:58.447558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9931ad-4f93-4fa1-9474-1ae9cec6f54f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:58.448031 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9931ad-4f93-4fa1-9474-1ae9cec6f54f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:58.448479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9931ad-4f93-4fa1-9474-1ae9cec6f54f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:58.489433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f612b64-ee4d-49f1-8045-2f2d518577d1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:58.529197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9931ad-4f93-4fa1-9474-1ae9cec6f54f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:58.534241 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9931ad-4f93-4fa1-9474-1ae9cec6f54f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:58.537098 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:58.543006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411df2e-7543-4362-87c1-02842d40b73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:58.543626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411df2e-7543-4362-87c1-02842d40b73e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:58.543900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411df2e-7543-4362-87c1-02842d40b73e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:58.544169 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d411df2e-7543-4362-87c1-02842d40b73e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:58.639134 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411df2e-7543-4362-87c1-02842d40b73e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:58.640536 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d411df2e-7543-4362-87c1-02842d40b73e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:58.663741 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 98/199] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:12:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:12:58.664674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9931ad-4f93-4fa1-9474-1ae9cec6f54f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:58.704006 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:58.768689 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 102/200] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:12:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:12:58.769891 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411df2e-7543-4362-87c1-02842d40b73e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:58.791621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9931ad-4f93-4fa1-9474-1ae9cec6f54f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:58.815682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:58.841793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfffe1-23c3-4df3-92e8-23e87a7fb004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:58.843109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfffe1-23c3-4df3-92e8-23e87a7fb004 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:58.843548 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfffe1-23c3-4df3-92e8-23e87a7fb004 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:58.844035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfffe1-23c3-4df3-92e8-23e87a7fb004 None admin] PATH_INFO: `/v3/users/273750f0431f44e8b0f7fc158b0b2832` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:58.845492 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dfffe1-23c3-4df3-92e8-23e87a7fb004 None admin] RBAC: Authorizing `identity:get_user(user_id=273750f0431f44e8b0f7fc158b0b2832)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:58.847838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dfffe1-23c3-4df3-92e8-23e87a7fb004 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:58.855847 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 99/201] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:12:58 2022] GET /identity/v3/users/273750f0431f44e8b0f7fc158b0b2832 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:58.855847 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfffe1-23c3-4df3-92e8-23e87a7fb004 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:58.907006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d411df2e-7543-4362-87c1-02842d40b73e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:58.959047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe30749-0620-4b10-85b5-51cce5df203c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:58.960762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe30749-0620-4b10-85b5-51cce5df203c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:58.961946 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe30749-0620-4b10-85b5-51cce5df203c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:58.963014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe30749-0620-4b10-85b5-51cce5df203c None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:58.964725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:58.965512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe30749-0620-4b10-85b5-51cce5df203c None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:58.968806 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe30749-0620-4b10-85b5-51cce5df203c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:58.972406 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 103/202] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:58 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:58.973190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe30749-0620-4b10-85b5-51cce5df203c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:58.974045 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfffe1-23c3-4df3-92e8-23e87a7fb004 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:59.028972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15d6d6-c7ca-4487-ae19-d2f5c7b97535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:59.030073 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15d6d6-c7ca-4487-ae19-d2f5c7b97535 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:59.030468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15d6d6-c7ca-4487-ae19-d2f5c7b97535 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:59.030830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15d6d6-c7ca-4487-ae19-d2f5c7b97535 None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:59.032484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15d6d6-c7ca-4487-ae19-d2f5c7b97535 None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:59.034032 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b15d6d6-c7ca-4487-ae19-d2f5c7b97535 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:59.036654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 100/203] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:58 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:59.037589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15d6d6-c7ca-4487-ae19-d2f5c7b97535 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:59.105823 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:59.110932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe30749-0620-4b10-85b5-51cce5df203c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:59.159269 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15d6d6-c7ca-4487-ae19-d2f5c7b97535 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:59.166747 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb2bc1-c857-4e96-8355-94099ca53211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:59.168211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb2bc1-c857-4e96-8355-94099ca53211 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:59.168522 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb2bc1-c857-4e96-8355-94099ca53211 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:59.168874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb2bc1-c857-4e96-8355-94099ca53211 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:59.197349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9413b-ba25-4f62-aab7-03eace9579b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:59.197962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9413b-ba25-4f62-aab7-03eace9579b7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:59.198250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9413b-ba25-4f62-aab7-03eace9579b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:59.198539 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c9413b-ba25-4f62-aab7-03eace9579b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:59.199262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9413b-ba25-4f62-aab7-03eace9579b7 None admin] RBAC: Adding query filter params (role.id=5a6ded16e6f54de78e356e7b984522fa, scope.project.id=7e93b768826c4821ad3dd853154e62b4, user.id=5878318c80a4470181cbe55d2c509015) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:59.199563 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9413b-ba25-4f62-aab7-03eace9579b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:59.201032 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c9413b-ba25-4f62-aab7-03eace9579b7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:59.207794 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 101/204] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:12:59 2022] GET /identity/v3/role_assignments?user.id=5878318c80a4470181cbe55d2c509015&scope.project.id=7e93b768826c4821ad3dd853154e62b4&role.id=5a6ded16e6f54de78e356e7b984522fa => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:59.208376 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9413b-ba25-4f62-aab7-03eace9579b7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:59.243907 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:59.262775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb2bc1-c857-4e96-8355-94099ca53211 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:59.264256 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdb2bc1-c857-4e96-8355-94099ca53211 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:59.323097 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c9413b-ba25-4f62-aab7-03eace9579b7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:59.358928 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7989e-6fe2-4ab5-8398-6c74f4d5359a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:59.359718 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7989e-6fe2-4ab5-8398-6c74f4d5359a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:59.360099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7989e-6fe2-4ab5-8398-6c74f4d5359a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:59.360470 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7989e-6fe2-4ab5-8398-6c74f4d5359a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:59.361344 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7989e-6fe2-4ab5-8398-6c74f4d5359a None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3, user.id=273750f0431f44e8b0f7fc158b0b2832) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:59.361756 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7989e-6fe2-4ab5-8398-6c74f4d5359a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:59.363437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f7989e-6fe2-4ab5-8398-6c74f4d5359a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:59.370256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 102/205] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:12:59 2022] GET /identity/v3/role_assignments?user.id=273750f0431f44e8b0f7fc158b0b2832&scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:59.374832 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:59.380112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 104/206] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:12:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:12:59.386122 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7989e-6fe2-4ab5-8398-6c74f4d5359a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:59.510631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:12:59.517493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f7989e-6fe2-4ab5-8398-6c74f4d5359a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:59.560606 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9ce747-753e-4d00-b956-d11353e028f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:59.561417 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ce747-753e-4d00-b956-d11353e028f8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:59.561751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ce747-753e-4d00-b956-d11353e028f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:59.562066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9ce747-753e-4d00-b956-d11353e028f8 None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:59.563547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ce747-753e-4d00-b956-d11353e028f8 None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:59.565454 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9ce747-753e-4d00-b956-d11353e028f8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:59.567628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 103/207] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:12:59 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:12:59.574354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb2bc1-c857-4e96-8355-94099ca53211 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:12:59.694757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdb2bc1-c857-4e96-8355-94099ca53211 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:12:59.737722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d62609-7d87-450d-8f4d-4538332f051c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:12:59.738418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d62609-7d87-450d-8f4d-4538332f051c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:12:59.738705 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d62609-7d87-450d-8f4d-4538332f051c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:12:59.738983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d62609-7d87-450d-8f4d-4538332f051c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:12:59.739731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d62609-7d87-450d-8f4d-4538332f051c None admin] RBAC: Adding query filter params (role.id=d123411b649d4acb8648d0c0b80aae6d, scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3, user.id=273750f0431f44e8b0f7fc158b0b2832) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:12:59.740048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d62609-7d87-450d-8f4d-4538332f051c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:12:59.741568 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d62609-7d87-450d-8f4d-4538332f051c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:12:59.748179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 105/208] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:12:59 2022] GET /identity/v3/role_assignments?user.id=273750f0431f44e8b0f7fc158b0b2832&scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3&role.id=d123411b649d4acb8648d0c0b80aae6d => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:00.139298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbb459-065a-4e9b-b00c-d836334302e1 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:00.139809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbb459-065a-4e9b-b00c-d836334302e1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:00.139882 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbb459-065a-4e9b-b00c-d836334302e1 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:00.141360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 104/209] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:00.146616 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d741f35-195c-43d8-a193-10f241a8eabc None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:00.146904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d741f35-195c-43d8-a193-10f241a8eabc None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:00.147203 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d741f35-195c-43d8-a193-10f241a8eabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:00.167415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6d741f35-195c-43d8-a193-10f241a8eabc None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:00.168888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6d741f35-195c-43d8-a193-10f241a8eabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:00.225467 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76073f-0dc9-4ec4-9d27-4028462f84d4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:00.225814 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76073f-0dc9-4ec4-9d27-4028462f84d4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:00.226014 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af76073f-0dc9-4ec4-9d27-4028462f84d4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:00.227607 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 105/210] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:00.232759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88b61d-f98f-40b8-8f9a-e9d4dce3cab6 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:00.233037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88b61d-f98f-40b8-8f9a-e9d4dce3cab6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:00.233309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d88b61d-f98f-40b8-8f9a-e9d4dce3cab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:00.242572 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:00.251786 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 106/211] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:00.253635 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8d88b61d-f98f-40b8-8f9a-e9d4dce3cab6 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:00.255140 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8d88b61d-f98f-40b8-8f9a-e9d4dce3cab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:00.322803 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 106/212] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:13:00.549664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d741f35-195c-43d8-a193-10f241a8eabc None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:00.602457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88b61d-f98f-40b8-8f9a-e9d4dce3cab6 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:00.665092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d741f35-195c-43d8-a193-10f241a8eabc None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:00.703098 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230fc19-2633-4955-b964-8b51f6d348d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:00.703686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230fc19-2633-4955-b964-8b51f6d348d3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:00.703963 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230fc19-2633-4955-b964-8b51f6d348d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:00.704226 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9230fc19-2633-4955-b964-8b51f6d348d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:00.711506 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:00.718035 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9230fc19-2633-4955-b964-8b51f6d348d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:00.723014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-9230fc19-2633-4955-b964-8b51f6d348d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:00.723014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:00.723014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:00.723014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:00.723014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:00.723014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:00.723014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:00.723014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:00.723014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:00.723014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:00.723014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:00.723014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:00.723014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:00.723014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:00.723014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:00.723014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:00.723864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:00.723864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:00.723864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:00.723864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:00.723864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:00.723864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:00.723864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:00.723864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:00.723864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:00.723864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:00.723864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:00.723864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:00.723864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:00.723864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:00.723864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:00.723864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:00.724634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:00.724634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:00.724634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:00.724634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:00.724634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:00.724634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:00.724634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:00.724634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:00.724634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:00.724634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:00.724634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:00.724634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:00.724634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 107/213] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:00.726049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d88b61d-f98f-40b8-8f9a-e9d4dce3cab6 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:00.730338 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230fc19-2633-4955-b964-8b51f6d348d3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:00.776969 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c84ba-3603-493a-b7b7-699e4dea15db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:00.777602 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c84ba-3603-493a-b7b7-699e4dea15db None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:00.777875 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c84ba-3603-493a-b7b7-699e4dea15db None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:00.778139 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c84ba-3603-493a-b7b7-699e4dea15db None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:00.780948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c84ba-3603-493a-b7b7-699e4dea15db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:00.782355 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c84ba-3603-493a-b7b7-699e4dea15db None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:00.783928 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 107/214] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:00.789481 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c84ba-3603-493a-b7b7-699e4dea15db None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:00.799679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:00.869874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9230fc19-2633-4955-b964-8b51f6d348d3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:00.904898 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c84ba-3603-493a-b7b7-699e4dea15db None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:00.920730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8e6fe-cfe8-4043-a694-78f66e913d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:00.922131 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8e6fe-cfe8-4043-a694-78f66e913d3f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:00.922963 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8e6fe-cfe8-4043-a694-78f66e913d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:00.923436 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e8e6fe-cfe8-4043-a694-78f66e913d3f None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:00.924573 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8e6fe-cfe8-4043-a694-78f66e913d3f None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:00.925501 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8e6fe-cfe8-4043-a694-78f66e913d3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:00.927222 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e8e6fe-cfe8-4043-a694-78f66e913d3f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:00.945243 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 108/215] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:00.946535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8e6fe-cfe8-4043-a694-78f66e913d3f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:00.950962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35beaec8-45dd-489f-9999-e5d99d90e158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:00.952428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35beaec8-45dd-489f-9999-e5d99d90e158 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:00.952428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35beaec8-45dd-489f-9999-e5d99d90e158 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:00.952428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35beaec8-45dd-489f-9999-e5d99d90e158 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:00.952834 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35beaec8-45dd-489f-9999-e5d99d90e158 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:00.954290 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35beaec8-45dd-489f-9999-e5d99d90e158 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:00.978368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 108/216] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:13:00 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:13:01.070621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e8e6fe-cfe8-4043-a694-78f66e913d3f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:01.089404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:01.110631 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3b0b6d-ef24-495c-be7e-3d5be86c9b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:01.111645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b0b6d-ef24-495c-be7e-3d5be86c9b6e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:01.112173 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b0b6d-ef24-495c-be7e-3d5be86c9b6e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:01.112641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3b0b6d-ef24-495c-be7e-3d5be86c9b6e None admin] PATH_INFO: `/v3/users/a34d05d5753d400d925d18a235605da8` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:01.128061 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b0b6d-ef24-495c-be7e-3d5be86c9b6e None admin] RBAC: Authorizing `identity:get_user(user_id=a34d05d5753d400d925d18a235605da8)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:01.129846 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3b0b6d-ef24-495c-be7e-3d5be86c9b6e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:01.132662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 109/217] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:13:00 2022] GET /identity/v3/users/a34d05d5753d400d925d18a235605da8 => generated 297 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:01.138536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35beaec8-45dd-489f-9999-e5d99d90e158 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:01.158199 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:01.249726 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35beaec8-45dd-489f-9999-e5d99d90e158 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:01.287173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d08d7-2735-4535-8a31-95b4f498d361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:01.288189 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d08d7-2735-4535-8a31-95b4f498d361 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:01.288693 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d08d7-2735-4535-8a31-95b4f498d361 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:01.289173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d08d7-2735-4535-8a31-95b4f498d361 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:01.364057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d08d7-2735-4535-8a31-95b4f498d361 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:01.366273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d08d7-2735-4535-8a31-95b4f498d361 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:01.383211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:01.431999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27039995-d0fe-4e45-9983-d54a663100d2 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:01.432631 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27039995-d0fe-4e45-9983-d54a663100d2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:01.433147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27039995-d0fe-4e45-9983-d54a663100d2 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:01.434726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 110/218] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:13:01.440142 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7c70b-2f24-4d46-a341-0616aa74bdf5 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:01.440697 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7c70b-2f24-4d46-a341-0616aa74bdf5 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:01.441220 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c7c70b-2f24-4d46-a341-0616aa74bdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:01.477558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 109/219] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:01.482966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-92c7c70b-2f24-4d46-a341-0616aa74bdf5 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:01.483171 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d08d7-2735-4535-8a31-95b4f498d361 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:01.484166 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-92c7c70b-2f24-4d46-a341-0616aa74bdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:01.602687 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d08d7-2735-4535-8a31-95b4f498d361 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:01.623192 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:01.643081 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baecfe7-b495-4ff8-a7b6-d2e3f68690f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:01.644329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baecfe7-b495-4ff8-a7b6-d2e3f68690f7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:01.644674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baecfe7-b495-4ff8-a7b6-d2e3f68690f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:01.645013 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baecfe7-b495-4ff8-a7b6-d2e3f68690f7 None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:01.646342 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baecfe7-b495-4ff8-a7b6-d2e3f68690f7 None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:01.648632 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baecfe7-b495-4ff8-a7b6-d2e3f68690f7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:01.650953 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 110/220] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:01 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:01.656833 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baecfe7-b495-4ff8-a7b6-d2e3f68690f7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:01.661831 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 111/221] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:01.745907 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c575b11-b527-4b37-b4da-66c591ba1ab6 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:01.746719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c575b11-b527-4b37-b4da-66c591ba1ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:01.746719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c575b11-b527-4b37-b4da-66c591ba1ab6 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:01.751919 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 112/222] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:13:01.755741 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28da224-a020-4def-a5ef-4af0c412f57a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:01.756029 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28da224-a020-4def-a5ef-4af0c412f57a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:01.756305 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28da224-a020-4def-a5ef-4af0c412f57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:01.781727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baecfe7-b495-4ff8-a7b6-d2e3f68690f7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:01.794563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e28da224-a020-4def-a5ef-4af0c412f57a None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:01.797708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e28da224-a020-4def-a5ef-4af0c412f57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:01.848106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1891354-4a9f-4d46-8ae0-4d1ea05f1ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:01.849128 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1891354-4a9f-4d46-8ae0-4d1ea05f1ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:01.849555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1891354-4a9f-4d46-8ae0-4d1ea05f1ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:01.849966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1891354-4a9f-4d46-8ae0-4d1ea05f1ee2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:01.851680 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1891354-4a9f-4d46-8ae0-4d1ea05f1ee2 None admin] RBAC: Adding query filter params (role.id=8d794ef8abb94809bd988c5db32c0037, scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3, user.id=a34d05d5753d400d925d18a235605da8) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:01.852132 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1891354-4a9f-4d46-8ae0-4d1ea05f1ee2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:01.853703 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1891354-4a9f-4d46-8ae0-4d1ea05f1ee2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:01.860267 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 111/223] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:13:01 2022] GET /identity/v3/role_assignments?user.id=a34d05d5753d400d925d18a235605da8&scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3&role.id=8d794ef8abb94809bd988c5db32c0037 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:01.929048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 113/224] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:01.994198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1891354-4a9f-4d46-8ae0-4d1ea05f1ee2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:02.047973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:02.109675 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1891354-4a9f-4d46-8ae0-4d1ea05f1ee2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:02.144486 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc3c69-a04a-4593-a1af-7e3fa91bfc3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:02.145112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc3c69-a04a-4593-a1af-7e3fa91bfc3e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:02.145388 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc3c69-a04a-4593-a1af-7e3fa91bfc3e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:02.145651 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cc3c69-a04a-4593-a1af-7e3fa91bfc3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:02.158036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cc3c69-a04a-4593-a1af-7e3fa91bfc3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:02.163398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d9cc3c69-a04a-4593-a1af-7e3fa91bfc3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:13:02.163398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:02.163398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:02.163398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:02.163398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:02.163398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:02.163398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:02.163398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:02.163398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:02.163398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:02.163398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:02.163398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:02.163398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:02.163398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:02.163398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:02.163398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:02.165165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:02.165165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:02.165165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:02.165165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:02.165165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:02.165165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:02.165165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:02.165165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:02.165165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:02.165165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:02.165165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:02.165165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:02.165165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:02.165165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:02.165165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:02.165165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:02.166153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:02.166153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:02.166153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:02.166153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:02.166153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:02.166153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:02.166153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:02.166153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:02.166153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:02.166153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:02.166153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:13:02.166153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:02.166153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 112/225] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:13:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:02.170379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28da224-a020-4def-a5ef-4af0c412f57a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:02.212526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:02.296269 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28da224-a020-4def-a5ef-4af0c412f57a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:02.324217 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc3c69-a04a-4593-a1af-7e3fa91bfc3e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:02.357154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddd219-4fc3-40b5-9c64-ba3f275fc856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:02.357772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddd219-4fc3-40b5-9c64-ba3f275fc856 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:02.358045 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddd219-4fc3-40b5-9c64-ba3f275fc856 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:02.358337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ddd219-4fc3-40b5-9c64-ba3f275fc856 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:02.359012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ddd219-4fc3-40b5-9c64-ba3f275fc856 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:02.359309 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ddd219-4fc3-40b5-9c64-ba3f275fc856 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:02.360700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ddd219-4fc3-40b5-9c64-ba3f275fc856 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:02.370789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 114/226] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:02.377170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddd219-4fc3-40b5-9c64-ba3f275fc856 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:02.492066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:02.505036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ddd219-4fc3-40b5-9c64-ba3f275fc856 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:02.552834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e8004-1e77-42ce-bb5c-0b2e37636109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:02.553518 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e8004-1e77-42ce-bb5c-0b2e37636109 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:02.553794 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e8004-1e77-42ce-bb5c-0b2e37636109 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:02.554063 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606e8004-1e77-42ce-bb5c-0b2e37636109 None admin] PATH_INFO: `/v3/users/273750f0431f44e8b0f7fc158b0b2832` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:02.555394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e8004-1e77-42ce-bb5c-0b2e37636109 None admin] RBAC: Authorizing `identity:get_user(user_id=273750f0431f44e8b0f7fc158b0b2832)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:02.556843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606e8004-1e77-42ce-bb5c-0b2e37636109 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:02.558846 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 115/227] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:13:02 2022] GET /identity/v3/users/273750f0431f44e8b0f7fc158b0b2832 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:02.564901 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e8004-1e77-42ce-bb5c-0b2e37636109 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:02.596232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cc3c69-a04a-4593-a1af-7e3fa91bfc3e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:02.611394 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:02.660110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bccf25-1a83-45fa-ba86-ff727b7ad103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:02.660757 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bccf25-1a83-45fa-ba86-ff727b7ad103 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:02.661036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bccf25-1a83-45fa-ba86-ff727b7ad103 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:02.661307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bccf25-1a83-45fa-ba86-ff727b7ad103 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:02.684805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bccf25-1a83-45fa-ba86-ff727b7ad103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:02.690098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f2bccf25-1a83-45fa-ba86-ff727b7ad103 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:02.690098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:02.690098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:02.690098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:02.690098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:02.690098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:02.690098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:02.690098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:02.690098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:02.690098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:02.690098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:02.690098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:02.690098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:02.690098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:02.690098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:02.690098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:02.691086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:02.691086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:02.691086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:02.691086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:02.691086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:02.691086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:02.691086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:02.691086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:02.691086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:02.691086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:02.691086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:02.691086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:02.691086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:02.691086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:02.691086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:02.691086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:02.691878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:02.691878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:02.691878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:02.691878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:02.691878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:02.691878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:02.691878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:02.691878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:02.691878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:02.691878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:02.691878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:02.691878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:02.691878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 113/228] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:13:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:13:02.700613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bccf25-1a83-45fa-ba86-ff727b7ad103 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:02.753776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606e8004-1e77-42ce-bb5c-0b2e37636109 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:02.809763 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4747af-b6ef-4e68-b0ae-69528bc81a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:02.810426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4747af-b6ef-4e68-b0ae-69528bc81a9e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:02.810704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4747af-b6ef-4e68-b0ae-69528bc81a9e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:02.810972 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4747af-b6ef-4e68-b0ae-69528bc81a9e None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:02.812209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4747af-b6ef-4e68-b0ae-69528bc81a9e None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:02.813642 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4747af-b6ef-4e68-b0ae-69528bc81a9e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:02.817564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 116/229] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:02 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:02.824860 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4747af-b6ef-4e68-b0ae-69528bc81a9e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:02.869075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bccf25-1a83-45fa-ba86-ff727b7ad103 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:02.903528 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:02.907678 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08516b-7dd2-4ae1-9b73-6ff770770176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:02.908294 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08516b-7dd2-4ae1-9b73-6ff770770176 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:02.908628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08516b-7dd2-4ae1-9b73-6ff770770176 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:02.908900 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da08516b-7dd2-4ae1-9b73-6ff770770176 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:02.909783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08516b-7dd2-4ae1-9b73-6ff770770176 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:02.910090 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08516b-7dd2-4ae1-9b73-6ff770770176 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:02.911494 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da08516b-7dd2-4ae1-9b73-6ff770770176 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:02.930510 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 114/230] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:02.936221 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08516b-7dd2-4ae1-9b73-6ff770770176 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:02.999621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:03.008758 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4747af-b6ef-4e68-b0ae-69528bc81a9e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:03.094985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4142b-21a2-4e94-bd38-acadd2fcc3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:03.095686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4142b-21a2-4e94-bd38-acadd2fcc3c0 None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:03.095792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da08516b-7dd2-4ae1-9b73-6ff770770176 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:03.095969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4142b-21a2-4e94-bd38-acadd2fcc3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:03.096259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe4142b-21a2-4e94-bd38-acadd2fcc3c0 None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3/users/273750f0431f44e8b0f7fc158b0b2832/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:03.103022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4142b-21a2-4e94-bd38-acadd2fcc3c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a83c130b5ab41c69c2d74715f3c92d3, user_id=273750f0431f44e8b0f7fc158b0b2832, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:03.104480 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe4142b-21a2-4e94-bd38-acadd2fcc3c0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:03.136621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaeb5f0-05f7-4247-9eb8-dfb62b921258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:03.137685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaeb5f0-05f7-4247-9eb8-dfb62b921258 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:03.138195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaeb5f0-05f7-4247-9eb8-dfb62b921258 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:03.138682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaeb5f0-05f7-4247-9eb8-dfb62b921258 None admin] PATH_INFO: `/v3/users/273750f0431f44e8b0f7fc158b0b2832` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:03.140123 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaeb5f0-05f7-4247-9eb8-dfb62b921258 None admin] RBAC: Authorizing `identity:get_user(user_id=273750f0431f44e8b0f7fc158b0b2832)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:03.140690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 117/231] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:13:02 2022] PUT /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3/users/273750f0431f44e8b0f7fc158b0b2832/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:13:03.142092 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaeb5f0-05f7-4247-9eb8-dfb62b921258 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:03.144419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 115/232] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:13:02 2022] GET /identity/v3/users/273750f0431f44e8b0f7fc158b0b2832 => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:03.151027 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4142b-21a2-4e94-bd38-acadd2fcc3c0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:03.301560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:03.313962 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe4142b-21a2-4e94-bd38-acadd2fcc3c0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:03.360917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6647c3a7-1d1d-47ec-be1f-fc866852e323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:03.361627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647c3a7-1d1d-47ec-be1f-fc866852e323 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:03.361895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647c3a7-1d1d-47ec-be1f-fc866852e323 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:03.362164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6647c3a7-1d1d-47ec-be1f-fc866852e323 None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:03.363373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647c3a7-1d1d-47ec-be1f-fc866852e323 None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:03.364772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6647c3a7-1d1d-47ec-be1f-fc866852e323 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:03.366914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 118/233] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:03 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:03.372358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaeb5f0-05f7-4247-9eb8-dfb62b921258 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:03.375406 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5f1db-8f6a-4c0f-b4ce-1fe9a7cfdb02 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:03.375684 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5f1db-8f6a-4c0f-b4ce-1fe9a7cfdb02 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:03.375953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5f1db-8f6a-4c0f-b4ce-1fe9a7cfdb02 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:03.377633 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 119/234] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:03.383768 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f00274-f2d1-4bce-8588-2046a66af53d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:03.384047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f00274-f2d1-4bce-8588-2046a66af53d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:03.384319 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f00274-f2d1-4bce-8588-2046a66af53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:03.404222 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-86f00274-f2d1-4bce-8588-2046a66af53d None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:03.405647 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-86f00274-f2d1-4bce-8588-2046a66af53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:03.510257 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:03.515171 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaeb5f0-05f7-4247-9eb8-dfb62b921258 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:03.533674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 120/235] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:13:03.579902 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28d846-f9ce-4148-a3f5-bb85d85fa51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:03.580494 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28d846-f9ce-4148-a3f5-bb85d85fa51b None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:03.580776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28d846-f9ce-4148-a3f5-bb85d85fa51b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:03.581052 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28d846-f9ce-4148-a3f5-bb85d85fa51b None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3/users/273750f0431f44e8b0f7fc158b0b2832/roles/d123411b649d4acb8648d0c0b80aae6d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:03.585798 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28d846-f9ce-4148-a3f5-bb85d85fa51b None admin] RBAC: Authorizing `identity:create_grant(project_id=5a83c130b5ab41c69c2d74715f3c92d3, user_id=273750f0431f44e8b0f7fc158b0b2832, role_id=d123411b649d4acb8648d0c0b80aae6d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:03.587279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28d846-f9ce-4148-a3f5-bb85d85fa51b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:03.607727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 116/236] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:13:03 2022] PUT /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3/users/273750f0431f44e8b0f7fc158b0b2832/roles/d123411b649d4acb8648d0c0b80aae6d => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:13:03.685032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d675d23-b363-4b55-bae6-69754fb1519f None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:03.685360 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d675d23-b363-4b55-bae6-69754fb1519f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:03.685595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d675d23-b363-4b55-bae6-69754fb1519f None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:03.687378 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 121/237] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:03.692870 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debff8f3-4ede-4d77-a4e4-d2381d3dc6a9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:03.693164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debff8f3-4ede-4d77-a4e4-d2381d3dc6a9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:03.693446 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debff8f3-4ede-4d77-a4e4-d2381d3dc6a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:03.719390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-debff8f3-4ede-4d77-a4e4-d2381d3dc6a9 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:03.720657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-debff8f3-4ede-4d77-a4e4-d2381d3dc6a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:03.810086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 117/238] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:03.911344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d675d23-b363-4b55-bae6-69754fb1519f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:04.044178 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:04.059951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d675d23-b363-4b55-bae6-69754fb1519f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:04.097873 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debff8f3-4ede-4d77-a4e4-d2381d3dc6a9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:04.124407 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed472ed4-5f4c-4616-a5c1-bac1157435bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:04.125059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed472ed4-5f4c-4616-a5c1-bac1157435bb None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:04.125336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed472ed4-5f4c-4616-a5c1-bac1157435bb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:04.125609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed472ed4-5f4c-4616-a5c1-bac1157435bb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:04.149396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed472ed4-5f4c-4616-a5c1-bac1157435bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:04.159587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ed472ed4-5f4c-4616-a5c1-bac1157435bb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:13:04.159587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:04.159587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:04.159587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:04.159587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:04.159587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:04.159587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:04.159587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:04.159587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:04.159587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:04.159587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:04.159587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:04.159587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:04.159587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:04.159587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:04.159587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:04.160502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:04.160502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:04.160502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:04.160502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:04.160502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:04.160502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:04.160502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:04.160502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:04.160502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:04.160502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:04.160502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:04.160502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:04.160502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:04.160502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:04.160502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:04.160502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:04.161273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:04.161273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:04.161273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:04.161273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:04.161273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:04.161273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:04.161273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:04.161273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:04.161273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:04.161273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:04.161273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:13:04.161273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:04.161908 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 122/239] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:04.170913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed472ed4-5f4c-4616-a5c1-bac1157435bb None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:04.323739 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:04.324140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed472ed4-5f4c-4616-a5c1-bac1157435bb None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:04.369996 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b80ded-2dd3-48ae-a87d-46531745a239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:04.371178 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b80ded-2dd3-48ae-a87d-46531745a239 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:04.371657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b80ded-2dd3-48ae-a87d-46531745a239 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:04.372139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b80ded-2dd3-48ae-a87d-46531745a239 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:04.373011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b80ded-2dd3-48ae-a87d-46531745a239 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:04.373520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b80ded-2dd3-48ae-a87d-46531745a239 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:04.375136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b80ded-2dd3-48ae-a87d-46531745a239 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:04.385332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debff8f3-4ede-4d77-a4e4-d2381d3dc6a9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:04.393629 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 123/240] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:04 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:04.399712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b018775-eaf8-4aee-bade-67a64a558c83 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:04.400012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b018775-eaf8-4aee-bade-67a64a558c83 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:04.400526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b018775-eaf8-4aee-bade-67a64a558c83 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:04.401964 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 124/241] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:04.402558 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b018775-eaf8-4aee-bade-67a64a558c83 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:04.435567 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def36557-ee0e-4418-af04-c5db224701c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:04.436300 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def36557-ee0e-4418-af04-c5db224701c6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:04.436581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def36557-ee0e-4418-af04-c5db224701c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:04.437309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def36557-ee0e-4418-af04-c5db224701c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:04.438121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def36557-ee0e-4418-af04-c5db224701c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:04.439720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def36557-ee0e-4418-af04-c5db224701c6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:04.441746 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 118/242] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:04.442559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:04.443157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea14032-79f0-42c5-afbf-a3e4b94287fd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:04.443535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea14032-79f0-42c5-afbf-a3e4b94287fd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:04.444022 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea14032-79f0-42c5-afbf-a3e4b94287fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:04.464258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-eea14032-79f0-42c5-afbf-a3e4b94287fd None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:04.465621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-eea14032-79f0-42c5-afbf-a3e4b94287fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:04.550359 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b018775-eaf8-4aee-bade-67a64a558c83 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:04.561357 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 119/243] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:04.562551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea14032-79f0-42c5-afbf-a3e4b94287fd None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:04.588480 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b9109-cef2-40b8-b688-e2586ac4be5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:04.589137 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b9109-cef2-40b8-b688-e2586ac4be5a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:04.589423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b9109-cef2-40b8-b688-e2586ac4be5a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:04.589663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b9109-cef2-40b8-b688-e2586ac4be5a None admin] PATH_INFO: `/v3/users/c8df7b14db4b40ae966a31239263eae5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:04.604744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b9109-cef2-40b8-b688-e2586ac4be5a None admin] RBAC: Authorizing `identity:get_user(user_id=c8df7b14db4b40ae966a31239263eae5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:04.606230 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b9109-cef2-40b8-b688-e2586ac4be5a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:04.608487 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 125/244] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:13:04 2022] GET /identity/v3/users/c8df7b14db4b40ae966a31239263eae5 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:04.615222 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b9109-cef2-40b8-b688-e2586ac4be5a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:04.659536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:04.678780 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea14032-79f0-42c5-afbf-a3e4b94287fd None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:04.716358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21b15e-cb63-4db6-84a5-62c0d08abb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:04.716955 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21b15e-cb63-4db6-84a5-62c0d08abb9d None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:04.717235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21b15e-cb63-4db6-84a5-62c0d08abb9d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:04.717507 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21b15e-cb63-4db6-84a5-62c0d08abb9d None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:04.718158 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf21b15e-cb63-4db6-84a5-62c0d08abb9d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:04.719642 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf21b15e-cb63-4db6-84a5-62c0d08abb9d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:04.746170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 120/245] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:13:04 2022] POST /identity/v3/users => generated 293 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:13:04.772977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:04.810495 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b9109-cef2-40b8-b688-e2586ac4be5a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:04.850736 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881eb348-9946-4937-805c-533168dccc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:04.851355 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881eb348-9946-4937-805c-533168dccc6b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:04.851629 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881eb348-9946-4937-805c-533168dccc6b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:04.851897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881eb348-9946-4937-805c-533168dccc6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:04.929742 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881eb348-9946-4937-805c-533168dccc6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:04.931302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881eb348-9946-4937-805c-533168dccc6b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:04.969411 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21b15e-cb63-4db6-84a5-62c0d08abb9d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:05.035882 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:05.091560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 126/246] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:05.097587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881eb348-9946-4937-805c-533168dccc6b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:05.125968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21b15e-cb63-4db6-84a5-62c0d08abb9d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:05.171370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcaf71-b931-4952-a57f-427aa0f9edc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:05.172500 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcaf71-b931-4952-a57f-427aa0f9edc2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:05.172976 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcaf71-b931-4952-a57f-427aa0f9edc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:05.173436 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abcaf71-b931-4952-a57f-427aa0f9edc2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:05.184637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:05.191157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abcaf71-b931-4952-a57f-427aa0f9edc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:05.200681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-9abcaf71-b931-4952-a57f-427aa0f9edc2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:05.200681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:05.200681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:05.200681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:05.200681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:05.200681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:05.200681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:05.200681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:05.200681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:05.200681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:05.200681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:05.200681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:05.200681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:05.200681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:05.200681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:05.200681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:05.201600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:05.201600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:05.201600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:05.201600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:05.201600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:05.201600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:05.201600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:05.201600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:05.201600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:05.201600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:05.201600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:05.201600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:05.201600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:05.201600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:05.201600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:05.201600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:05.202412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:05.202412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:05.202412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:05.202412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:05.202412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:05.202412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:05.202412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:05.202412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:05.202412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:05.202412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:05.202412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:05.202412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:05.205125 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 121/247] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:13:05.211718 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcaf71-b931-4952-a57f-427aa0f9edc2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:05.223775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881eb348-9946-4937-805c-533168dccc6b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:05.269324 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977dbd5-24a5-472c-872e-a43b0a50ac12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:05.270496 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977dbd5-24a5-472c-872e-a43b0a50ac12 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:05.270974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977dbd5-24a5-472c-872e-a43b0a50ac12 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:05.271513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e977dbd5-24a5-472c-872e-a43b0a50ac12 None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:05.272859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977dbd5-24a5-472c-872e-a43b0a50ac12 None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:05.277525 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e977dbd5-24a5-472c-872e-a43b0a50ac12 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:05.277525 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 127/248] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:05 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:05.283188 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977dbd5-24a5-472c-872e-a43b0a50ac12 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:05.347917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:05.367281 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abcaf71-b931-4952-a57f-427aa0f9edc2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:05.411232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef489d7f-1c8a-4272-a771-f2ffffbb8391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:05.412705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef489d7f-1c8a-4272-a771-f2ffffbb8391 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:05.413188 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef489d7f-1c8a-4272-a771-f2ffffbb8391 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:05.413809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef489d7f-1c8a-4272-a771-f2ffffbb8391 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:05.414831 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef489d7f-1c8a-4272-a771-f2ffffbb8391 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:05.415347 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef489d7f-1c8a-4272-a771-f2ffffbb8391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:05.417681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef489d7f-1c8a-4272-a771-f2ffffbb8391 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:05.427680 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 122/249] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:05.436782 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef489d7f-1c8a-4272-a771-f2ffffbb8391 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:05.439635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e977dbd5-24a5-472c-872e-a43b0a50ac12 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:05.480326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f3026-31ed-4292-9961-dcd1d01b3634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:05.481442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f3026-31ed-4292-9961-dcd1d01b3634 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:05.481918 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f3026-31ed-4292-9961-dcd1d01b3634 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:05.482436 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33f3026-31ed-4292-9961-dcd1d01b3634 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:05.483424 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f3026-31ed-4292-9961-dcd1d01b3634 None admin] RBAC: Adding query filter params (role.id=5a6ded16e6f54de78e356e7b984522fa, scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3, user.id=c8df7b14db4b40ae966a31239263eae5) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:05.483944 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f3026-31ed-4292-9961-dcd1d01b3634 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:05.485638 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33f3026-31ed-4292-9961-dcd1d01b3634 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:05.493326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 128/250] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:13:05 2022] GET /identity/v3/role_assignments?user.id=c8df7b14db4b40ae966a31239263eae5&scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3&role.id=5a6ded16e6f54de78e356e7b984522fa => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:05.524522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:05.575000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef489d7f-1c8a-4272-a771-f2ffffbb8391 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:05.616148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c08a34-d177-4a98-8779-8025c0ad5546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:05.616886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c08a34-d177-4a98-8779-8025c0ad5546 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:05.617170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c08a34-d177-4a98-8779-8025c0ad5546 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:05.617442 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c08a34-d177-4a98-8779-8025c0ad5546 None admin] PATH_INFO: `/v3/users/a34d05d5753d400d925d18a235605da8` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:05.618851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c08a34-d177-4a98-8779-8025c0ad5546 None admin] RBAC: Authorizing `identity:get_user(user_id=a34d05d5753d400d925d18a235605da8)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:05.620360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c08a34-d177-4a98-8779-8025c0ad5546 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:05.622093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 123/251] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:13:05 2022] GET /identity/v3/users/a34d05d5753d400d925d18a235605da8 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:05.628433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f3026-31ed-4292-9961-dcd1d01b3634 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:05.673529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5fbcc-26ef-416d-ae29-5047ace2d6f4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:05.674712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5fbcc-26ef-416d-ae29-5047ace2d6f4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:05.674712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5fbcc-26ef-416d-ae29-5047ace2d6f4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:05.676944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 124/252] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:05.685282 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fc27b-6711-44bb-9cac-6566e5e8336e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:05.685781 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fc27b-6711-44bb-9cac-6566e5e8336e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:05.686666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8fc27b-6711-44bb-9cac-6566e5e8336e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:05.738144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6f8fc27b-6711-44bb-9cac-6566e5e8336e None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:05.740383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6f8fc27b-6711-44bb-9cac-6566e5e8336e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:05.858498 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33f3026-31ed-4292-9961-dcd1d01b3634 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:05.860334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:05.926238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 125/253] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:05.943558 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bfd3df-31d2-4fcc-b9d7-1d84686246d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:05.944926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bfd3df-31d2-4fcc-b9d7-1d84686246d0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:05.945351 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bfd3df-31d2-4fcc-b9d7-1d84686246d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:05.945765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bfd3df-31d2-4fcc-b9d7-1d84686246d0 None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:05.950999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bfd3df-31d2-4fcc-b9d7-1d84686246d0 None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:05.953196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bfd3df-31d2-4fcc-b9d7-1d84686246d0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:05.955440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 129/254] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:05 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:05.963163 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fc27b-6711-44bb-9cac-6566e5e8336e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:06.045526 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:06.095297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8fc27b-6711-44bb-9cac-6566e5e8336e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:06.124701 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6bfec-86d6-4125-b580-7be3f041c9ff None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:06.125033 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6bfec-86d6-4125-b580-7be3f041c9ff None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:06.125254 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff6bfec-86d6-4125-b580-7be3f041c9ff None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:06.126905 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 130/255] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:13:06.133433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237a422-1e28-47e8-a37d-53adaa9ee381 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:06.133777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237a422-1e28-47e8-a37d-53adaa9ee381 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:06.134058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3237a422-1e28-47e8-a37d-53adaa9ee381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:06.144260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ef8d8-02ab-4e1f-9d02-260c445dd01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:06.144938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ef8d8-02ab-4e1f-9d02-260c445dd01a None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:06.145218 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ef8d8-02ab-4e1f-9d02-260c445dd01a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:06.145489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ef8d8-02ab-4e1f-9d02-260c445dd01a None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3/users/a34d05d5753d400d925d18a235605da8/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:06.151813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ef8d8-02ab-4e1f-9d02-260c445dd01a None admin] RBAC: Authorizing `identity:create_grant(project_id=5a83c130b5ab41c69c2d74715f3c92d3, user_id=a34d05d5753d400d925d18a235605da8, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:06.153277 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ef8d8-02ab-4e1f-9d02-260c445dd01a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:06.175002 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3237a422-1e28-47e8-a37d-53adaa9ee381 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:06.176203 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3237a422-1e28-47e8-a37d-53adaa9ee381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:06.176810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 126/256] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:13:05 2022] PUT /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3/users/a34d05d5753d400d925d18a235605da8/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:13:06.296915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 131/257] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:13:06.458500 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ef8d8-02ab-4e1f-9d02-260c445dd01a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:06.588673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ef8d8-02ab-4e1f-9d02-260c445dd01a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:06.615821 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:06.639454 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b97381-c5c3-4056-af07-e11bee8211c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:06.640111 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b97381-c5c3-4056-af07-e11bee8211c7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:06.640396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b97381-c5c3-4056-af07-e11bee8211c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:06.640661 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b97381-c5c3-4056-af07-e11bee8211c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:06.658542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b97381-c5c3-4056-af07-e11bee8211c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:06.663623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-32b97381-c5c3-4056-af07-e11bee8211c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:13:06.663623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:06.663623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:06.663623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:06.663623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:06.663623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:06.663623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:06.663623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:06.663623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:06.663623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:06.663623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:06.663623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:06.663623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:06.663623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:06.663623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:06.663623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:06.667044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:06.667044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:06.667044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:06.667044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:06.667044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:06.667044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:06.667044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:06.667044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:06.667044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:06.667044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:06.667044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:06.667044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:06.667044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:06.667044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:06.667044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:06.667044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:06.670101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:06.670101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:06.670101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:06.670101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:06.670101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:06.670101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:06.670101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:06.670101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:06.670101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:06.670101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:06.670101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:13:06.670101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:06.670101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 127/258] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:13:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:06.672073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237a422-1e28-47e8-a37d-53adaa9ee381 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:06.685895 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b97381-c5c3-4056-af07-e11bee8211c7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:06.908665 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b97381-c5c3-4056-af07-e11bee8211c7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:06.938101 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3237a422-1e28-47e8-a37d-53adaa9ee381 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:06.942438 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:06.973935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d2a34-aa15-4e2b-9fe9-e04b5574b7eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:06.974782 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d2a34-aa15-4e2b-9fe9-e04b5574b7eb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:06.975148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d2a34-aa15-4e2b-9fe9-e04b5574b7eb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:06.975939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27d2a34-aa15-4e2b-9fe9-e04b5574b7eb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:06.990692 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27d2a34-aa15-4e2b-9fe9-e04b5574b7eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:07.003666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e27d2a34-aa15-4e2b-9fe9-e04b5574b7eb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:07.003666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:07.003666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:07.003666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:07.003666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:07.003666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:07.003666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:07.003666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:07.003666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:07.003666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:07.003666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:07.003666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:07.003666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:07.003666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:07.003666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:07.003666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:07.004597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:07.004597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:07.004597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:07.004597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:07.004597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:07.004597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:07.004597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:07.004597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:07.004597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:07.004597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:07.004597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:07.004597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:07.004597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:07.004597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:07.004597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:07.004597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:07.005386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:07.005386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:07.005386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:07.005386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:07.005386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:07.005386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:07.005386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:07.005386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:07.005386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:07.005386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:07.005386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:07.005386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:07.005386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 128/259] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:13:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:07.015109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d2a34-aa15-4e2b-9fe9-e04b5574b7eb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:07.031579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a4971-6993-4a67-ab80-61676ffd9ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:07.032210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a4971-6993-4a67-ab80-61676ffd9ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:07.032472 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a4971-6993-4a67-ab80-61676ffd9ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:07.032844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a4971-6993-4a67-ab80-61676ffd9ed9 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:07.033755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a4971-6993-4a67-ab80-61676ffd9ed9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:07.034056 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a4971-6993-4a67-ab80-61676ffd9ed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:07.035729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281a4971-6993-4a67-ab80-61676ffd9ed9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:07.059693 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 132/260] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:07.065801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a4971-6993-4a67-ab80-61676ffd9ed9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:07.089444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:07.185739 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27d2a34-aa15-4e2b-9fe9-e04b5574b7eb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:07.237019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fcec3-f344-49b5-bb2a-9fa3c5e9b830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:07.238078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fcec3-f344-49b5-bb2a-9fa3c5e9b830 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:07.238485 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fcec3-f344-49b5-bb2a-9fa3c5e9b830 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:07.239817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149fcec3-f344-49b5-bb2a-9fa3c5e9b830 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:07.240608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149fcec3-f344-49b5-bb2a-9fa3c5e9b830 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:07.241038 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149fcec3-f344-49b5-bb2a-9fa3c5e9b830 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:07.242557 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149fcec3-f344-49b5-bb2a-9fa3c5e9b830 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:07.251903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 129/261] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:07.259719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fcec3-f344-49b5-bb2a-9fa3c5e9b830 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:07.318525 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a4971-6993-4a67-ab80-61676ffd9ed9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:07.382818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69176ef1-2896-45ea-9484-d8e72a8cf19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:07.383581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69176ef1-2896-45ea-9484-d8e72a8cf19f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:07.383858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69176ef1-2896-45ea-9484-d8e72a8cf19f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:07.384128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69176ef1-2896-45ea-9484-d8e72a8cf19f None admin] PATH_INFO: `/v3/users/273750f0431f44e8b0f7fc158b0b2832` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:07.386930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69176ef1-2896-45ea-9484-d8e72a8cf19f None admin] RBAC: Authorizing `identity:get_user(user_id=273750f0431f44e8b0f7fc158b0b2832)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:07.389423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69176ef1-2896-45ea-9484-d8e72a8cf19f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:07.395145 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 133/262] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:13:07 2022] GET /identity/v3/users/273750f0431f44e8b0f7fc158b0b2832 => generated 283 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:07.403870 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149fcec3-f344-49b5-bb2a-9fa3c5e9b830 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:07.404950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69176ef1-2896-45ea-9484-d8e72a8cf19f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:07.410501 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:07.462874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:07.480338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5f25b-128d-4939-9609-172f8fb4f79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:07.480984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5f25b-128d-4939-9609-172f8fb4f79a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:07.481260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5f25b-128d-4939-9609-172f8fb4f79a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:07.481524 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5f25b-128d-4939-9609-172f8fb4f79a None admin] PATH_INFO: `/v3/users/273750f0431f44e8b0f7fc158b0b2832` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:07.482688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5f25b-128d-4939-9609-172f8fb4f79a None admin] RBAC: Authorizing `identity:get_user(user_id=273750f0431f44e8b0f7fc158b0b2832)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:07.484153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5f25b-128d-4939-9609-172f8fb4f79a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:07.485905 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 130/263] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:13:07 2022] GET /identity/v3/users/273750f0431f44e8b0f7fc158b0b2832 => generated 283 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:07.492062 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5f25b-128d-4939-9609-172f8fb4f79a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:07.544229 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69176ef1-2896-45ea-9484-d8e72a8cf19f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:07.589894 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0c8e4-de79-45fb-867c-7575465ecc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:07.590726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0c8e4-de79-45fb-867c-7575465ecc55 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:07.591004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0c8e4-de79-45fb-867c-7575465ecc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:07.591273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf0c8e4-de79-45fb-867c-7575465ecc55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:07.666194 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5f25b-128d-4939-9609-172f8fb4f79a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:07.673955 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0c8e4-de79-45fb-867c-7575465ecc55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:07.675473 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf0c8e4-de79-45fb-867c-7575465ecc55 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:07.724322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e123ed0-0d04-4158-80f3-d8f2548d8acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:07.724869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e123ed0-0d04-4158-80f3-d8f2548d8acb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:07.725174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e123ed0-0d04-4158-80f3-d8f2548d8acb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:07.725446 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e123ed0-0d04-4158-80f3-d8f2548d8acb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:07.739794 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:07.810750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:07.880133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 134/264] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:07.887816 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e123ed0-0d04-4158-80f3-d8f2548d8acb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:07.890507 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e123ed0-0d04-4158-80f3-d8f2548d8acb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:07.899498 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0c8e4-de79-45fb-867c-7575465ecc55 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:08.024159 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf0c8e4-de79-45fb-867c-7575465ecc55 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:08.042862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 131/265] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:08.044202 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8695c21-404d-49e2-8027-4e7d90315298 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:08.044483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8695c21-404d-49e2-8027-4e7d90315298 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:08.044746 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8695c21-404d-49e2-8027-4e7d90315298 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:08.045974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 132/266] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:08.051570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4dd08-c5d0-465f-b364-2ea03259d4ad None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:08.051847 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4dd08-c5d0-465f-b364-2ea03259d4ad None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:08.052117 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e4dd08-c5d0-465f-b364-2ea03259d4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:08.066688 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:08.077278 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948315e3-4899-4231-99e9-2d780f870c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:08.078237 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948315e3-4899-4231-99e9-2d780f870c91 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:08.078735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948315e3-4899-4231-99e9-2d780f870c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:08.079196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948315e3-4899-4231-99e9-2d780f870c91 None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:08.082887 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948315e3-4899-4231-99e9-2d780f870c91 None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:08.083238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-63e4dd08-c5d0-465f-b364-2ea03259d4ad None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:08.084473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-63e4dd08-c5d0-465f-b364-2ea03259d4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:08.085573 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948315e3-4899-4231-99e9-2d780f870c91 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:08.087778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 135/267] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:07 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:08.088757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948315e3-4899-4231-99e9-2d780f870c91 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:08.185073 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:08.197098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 133/268] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:08.197367 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4dd08-c5d0-465f-b364-2ea03259d4ad None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:08.210420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948315e3-4899-4231-99e9-2d780f870c91 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:08.257637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101411fb-e815-4b1e-871c-111ecce82b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:08.258684 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101411fb-e815-4b1e-871c-111ecce82b93 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:08.259165 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101411fb-e815-4b1e-871c-111ecce82b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:08.259653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101411fb-e815-4b1e-871c-111ecce82b93 None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:08.266916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101411fb-e815-4b1e-871c-111ecce82b93 None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:08.268592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101411fb-e815-4b1e-871c-111ecce82b93 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:08.270805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 136/269] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:08 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:08.277115 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101411fb-e815-4b1e-871c-111ecce82b93 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:08.350513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e4dd08-c5d0-465f-b364-2ea03259d4ad None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:08.411873 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb452a29-9e24-4df6-9671-6a750666f8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:08.412592 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb452a29-9e24-4df6-9671-6a750666f8b0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:08.413001 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb452a29-9e24-4df6-9671-6a750666f8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:08.413304 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb452a29-9e24-4df6-9671-6a750666f8b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:08.414205 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb452a29-9e24-4df6-9671-6a750666f8b0 None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3, user.id=273750f0431f44e8b0f7fc158b0b2832) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:08.414536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb452a29-9e24-4df6-9671-6a750666f8b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:08.415980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb452a29-9e24-4df6-9671-6a750666f8b0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:08.420043 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:08.422904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 134/270] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:13:08 2022] GET /identity/v3/role_assignments?user.id=273750f0431f44e8b0f7fc158b0b2832&scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:08.435805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ba8b4-d4ed-4ba5-ba2c-623cbb97db72 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:08.436153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ba8b4-d4ed-4ba5-ba2c-623cbb97db72 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:08.436350 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247ba8b4-d4ed-4ba5-ba2c-623cbb97db72 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:08.437696 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 135/271] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:08 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:08.448327 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236a36b-07c3-4da1-9215-527e9f2211bc None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:08.448632 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236a36b-07c3-4da1-9215-527e9f2211bc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:08.448962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c236a36b-07c3-4da1-9215-527e9f2211bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:08.479691 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c236a36b-07c3-4da1-9215-527e9f2211bc None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:08.480894 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c236a36b-07c3-4da1-9215-527e9f2211bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:08.513775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101411fb-e815-4b1e-871c-111ecce82b93 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:08.588711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33f0f3-0148-4eec-aef7-b4417fac54ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:08.589740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33f0f3-0148-4eec-aef7-b4417fac54ef None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:08.590313 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33f0f3-0148-4eec-aef7-b4417fac54ef None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:08.590731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33f0f3-0148-4eec-aef7-b4417fac54ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:08.591486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33f0f3-0148-4eec-aef7-b4417fac54ef None admin] RBAC: Adding query filter params (role.id=d123411b649d4acb8648d0c0b80aae6d, scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3, user.id=273750f0431f44e8b0f7fc158b0b2832) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:08.591792 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33f0f3-0148-4eec-aef7-b4417fac54ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:08.593353 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33f0f3-0148-4eec-aef7-b4417fac54ef None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:08.595284 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:08.604127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 137/272] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:13:08 2022] GET /identity/v3/role_assignments?user.id=273750f0431f44e8b0f7fc158b0b2832&scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3&role.id=d123411b649d4acb8648d0c0b80aae6d => generated 608 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:08.605273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33f0f3-0148-4eec-aef7-b4417fac54ef None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:08.617924 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 136/273] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:13:08.619885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53a42e-8d12-4f34-82b3-1aa6582b5804 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:08.620167 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53a42e-8d12-4f34-82b3-1aa6582b5804 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:08.620438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53a42e-8d12-4f34-82b3-1aa6582b5804 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:08.624630 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 137/274] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:08.630736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e8e05-47c7-43d0-9065-9b3c6503ecb5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:08.631146 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e8e05-47c7-43d0-9065-9b3c6503ecb5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:08.631865 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e8e05-47c7-43d0-9065-9b3c6503ecb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:08.671645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-326e8e05-47c7-43d0-9065-9b3c6503ecb5 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:08.675483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-326e8e05-47c7-43d0-9065-9b3c6503ecb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:08.785824 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 138/275] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:08.819580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33f0f3-0148-4eec-aef7-b4417fac54ef None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:08.857944 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:08.864849 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b6445-0aa5-4798-919e-2d16b18a59d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:08.865636 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b6445-0aa5-4798-919e-2d16b18a59d1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:08.866001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b6445-0aa5-4798-919e-2d16b18a59d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:08.866998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b6445-0aa5-4798-919e-2d16b18a59d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:08.880487 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b6445-0aa5-4798-919e-2d16b18a59d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:08.885858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-c95b6445-0aa5-4798-919e-2d16b18a59d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:08.885858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:08.885858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:08.885858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:08.885858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:08.885858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:08.885858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:08.885858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:08.885858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:08.885858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:08.885858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:08.885858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:08.885858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:08.885858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:08.885858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:08.885858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:08.886740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:08.886740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:08.886740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:08.886740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:08.886740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:08.886740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:08.886740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:08.886740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:08.886740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:08.886740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:08.886740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:08.886740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:08.886740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:08.886740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:08.886740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:08.886740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:08.887512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:08.887512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:08.887512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:08.887512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:08.887512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:08.887512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:08.887512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:08.887512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:08.887512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:08.887512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:08.887512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:08.887512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:08.887512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 138/276] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:08.892874 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e8e05-47c7-43d0-9065-9b3c6503ecb5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:08.981031 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b6445-0aa5-4798-919e-2d16b18a59d1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:09.002918 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e8e05-47c7-43d0-9065-9b3c6503ecb5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:09.033432 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:09.048096 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa30587-1f0b-483f-9258-cdeb67c8faa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:09.049062 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa30587-1f0b-483f-9258-cdeb67c8faa8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:09.049555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa30587-1f0b-483f-9258-cdeb67c8faa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:09.050020 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa30587-1f0b-483f-9258-cdeb67c8faa8 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:09.050911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa30587-1f0b-483f-9258-cdeb67c8faa8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:09.051408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa30587-1f0b-483f-9258-cdeb67c8faa8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:09.053039 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa30587-1f0b-483f-9258-cdeb67c8faa8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:09.066593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 139/277] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:09.069332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa30587-1f0b-483f-9258-cdeb67c8faa8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:09.112821 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b6445-0aa5-4798-919e-2d16b18a59d1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:09.151784 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094a7db-f323-4534-b40e-277268bbb6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:09.152416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094a7db-f323-4534-b40e-277268bbb6cd None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:09.152695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094a7db-f323-4534-b40e-277268bbb6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:09.152967 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094a7db-f323-4534-b40e-277268bbb6cd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:09.163901 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094a7db-f323-4534-b40e-277268bbb6cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:09.169112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-5094a7db-f323-4534-b40e-277268bbb6cd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:13:09.169112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:09.169112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:09.169112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:09.169112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:09.169112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:09.169112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:09.169112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:09.169112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:09.169112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:09.169112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:09.169112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:09.169112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:09.169112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:09.169112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:09.169112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:09.169986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:09.169986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:09.169986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:09.169986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:09.169986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:09.169986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:09.169986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:09.169986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:09.169986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:09.169986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:09.169986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:09.169986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:09.169986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:09.169986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:09.169986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:09.169986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:09.175301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:09.175301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:09.175301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:09.175301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:09.175301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:09.175301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:09.175301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:09.175301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:09.175301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:09.175301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:09.175301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:13:09.175301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:09.175301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 139/278] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:09.175301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094a7db-f323-4534-b40e-277268bbb6cd None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:09.190755 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa30587-1f0b-483f-9258-cdeb67c8faa8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:09.227339 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:09.228113 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3c1e0-2dcc-44bb-95d1-4375cebe83a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:09.228733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3c1e0-2dcc-44bb-95d1-4375cebe83a1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:09.229095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3c1e0-2dcc-44bb-95d1-4375cebe83a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:09.229401 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b3c1e0-2dcc-44bb-95d1-4375cebe83a1 None admin] PATH_INFO: `/v3/users/364a4222d88143d6aade99b011dd76c2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:09.244216 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3c1e0-2dcc-44bb-95d1-4375cebe83a1 None admin] RBAC: Authorizing `identity:get_user(user_id=364a4222d88143d6aade99b011dd76c2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:09.246270 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b3c1e0-2dcc-44bb-95d1-4375cebe83a1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:09.248195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 140/279] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:13:09 2022] GET /identity/v3/users/364a4222d88143d6aade99b011dd76c2 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:09.249050 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3c1e0-2dcc-44bb-95d1-4375cebe83a1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:09.288222 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094a7db-f323-4534-b40e-277268bbb6cd None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:09.298562 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:09.324400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf27ed-6c4b-49ca-96ca-8c9cf87f3650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:09.325026 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf27ed-6c4b-49ca-96ca-8c9cf87f3650 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:09.325904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf27ed-6c4b-49ca-96ca-8c9cf87f3650 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:09.325904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cf27ed-6c4b-49ca-96ca-8c9cf87f3650 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:09.326751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf27ed-6c4b-49ca-96ca-8c9cf87f3650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:09.328183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cf27ed-6c4b-49ca-96ca-8c9cf87f3650 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:09.330265 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 140/280] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:09.330519 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf27ed-6c4b-49ca-96ca-8c9cf87f3650 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:09.361339 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b3c1e0-2dcc-44bb-95d1-4375cebe83a1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:09.397902 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b62dcf-1531-42d8-8f9d-9894406830ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:09.398526 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b62dcf-1531-42d8-8f9d-9894406830ff None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:09.398962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b62dcf-1531-42d8-8f9d-9894406830ff None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:09.399066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b62dcf-1531-42d8-8f9d-9894406830ff None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:09.399712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b62dcf-1531-42d8-8f9d-9894406830ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:09.400066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b62dcf-1531-42d8-8f9d-9894406830ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:09.401685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b62dcf-1531-42d8-8f9d-9894406830ff None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:09.414509 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 141/281] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:09 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:09.414509 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b62dcf-1531-42d8-8f9d-9894406830ff None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:09.444875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cf27ed-6c4b-49ca-96ca-8c9cf87f3650 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:09.462894 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:09.480780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cea4a-dcde-45e9-ae6e-04e676d34b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:09.481389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cea4a-dcde-45e9-ae6e-04e676d34b27 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:09.481663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cea4a-dcde-45e9-ae6e-04e676d34b27 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:09.481930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4cea4a-dcde-45e9-ae6e-04e676d34b27 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:09.482671 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cea4a-dcde-45e9-ae6e-04e676d34b27 None admin] RBAC: Adding query filter params (role.id=8d794ef8abb94809bd988c5db32c0037, user.id=364a4222d88143d6aade99b011dd76c2, scope.system=all) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:09.482974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cea4a-dcde-45e9-ae6e-04e676d34b27 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:09.484400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4cea4a-dcde-45e9-ae6e-04e676d34b27 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:09.489220 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 141/282] 10.222.0.76 () {56 vars in 1308 bytes} [Wed Aug 24 15:13:09 2022] GET /identity/v3/role_assignments?user.id=364a4222d88143d6aade99b011dd76c2&scope.system=all&role.id=8d794ef8abb94809bd988c5db32c0037 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:09.489986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cea4a-dcde-45e9-ae6e-04e676d34b27 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:09.536013 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b62dcf-1531-42d8-8f9d-9894406830ff None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:09.544832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:09.576364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94600689-22b7-4c94-947a-4595b5f6e8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:09.576940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94600689-22b7-4c94-947a-4595b5f6e8de None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:09.577216 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94600689-22b7-4c94-947a-4595b5f6e8de None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:09.577483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94600689-22b7-4c94-947a-4595b5f6e8de None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:09.578151 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94600689-22b7-4c94-947a-4595b5f6e8de None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:09.579843 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94600689-22b7-4c94-947a-4595b5f6e8de None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:09.607856 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 142/283] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:13:09 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:13:09.610682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d4302-d5ff-4ffe-a50a-01ae4ea99aa9 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:09.611412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d4302-d5ff-4ffe-a50a-01ae4ea99aa9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:09.612083 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d4302-d5ff-4ffe-a50a-01ae4ea99aa9 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:09.614525 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 143/284] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:09.622295 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1a9c3-51be-4b18-bea2-e0be46ce4efb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:09.623070 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1a9c3-51be-4b18-bea2-e0be46ce4efb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:09.624396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1a9c3-51be-4b18-bea2-e0be46ce4efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:09.634116 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4cea4a-dcde-45e9-ae6e-04e676d34b27 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:09.651896 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a8d1a9c3-51be-4b18-bea2-e0be46ce4efb None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:09.653574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a8d1a9c3-51be-4b18-bea2-e0be46ce4efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:09.686472 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cd46a-9725-4d55-90af-931d53ea27e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:09.687464 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cd46a-9725-4d55-90af-931d53ea27e8 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:09.687782 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cd46a-9725-4d55-90af-931d53ea27e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:09.688600 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5cd46a-9725-4d55-90af-931d53ea27e8 None admin] PATH_INFO: `/v3/users/c8df7b14db4b40ae966a31239263eae5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:09.690272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cd46a-9725-4d55-90af-931d53ea27e8 None admin] RBAC: Authorizing `identity:get_user(user_id=c8df7b14db4b40ae966a31239263eae5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:09.692935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5cd46a-9725-4d55-90af-931d53ea27e8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:09.696614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 142/285] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:13:09 2022] GET /identity/v3/users/c8df7b14db4b40ae966a31239263eae5 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:09.703832 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cd46a-9725-4d55-90af-931d53ea27e8 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:09.743145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 144/286] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:09.774817 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:09.829501 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5cd46a-9725-4d55-90af-931d53ea27e8 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:09.866868 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa0751-bc2a-463d-adbd-a97f424aa58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:09.867500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa0751-bc2a-463d-adbd-a97f424aa58a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:09.867775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa0751-bc2a-463d-adbd-a97f424aa58a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:09.868041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa0751-bc2a-463d-adbd-a97f424aa58a None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:09.869176 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa0751-bc2a-463d-adbd-a97f424aa58a None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:09.870672 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa0751-bc2a-463d-adbd-a97f424aa58a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:09.872675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 143/287] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:09 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:09.878133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1a9c3-51be-4b18-bea2-e0be46ce4efb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:09.977801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:09.990979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1a9c3-51be-4b18-bea2-e0be46ce4efb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:10.031647 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79b0d6-30d2-40ce-8f37-cdc273fb40fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:10.032166 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa0751-bc2a-463d-adbd-a97f424aa58a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:10.032581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79b0d6-30d2-40ce-8f37-cdc273fb40fa None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:10.032960 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79b0d6-30d2-40ce-8f37-cdc273fb40fa None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:10.033522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c79b0d6-30d2-40ce-8f37-cdc273fb40fa None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3/users/c8df7b14db4b40ae966a31239263eae5/roles/5a6ded16e6f54de78e356e7b984522fa` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:10.035777 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c79b0d6-30d2-40ce-8f37-cdc273fb40fa None admin] RBAC: Authorizing `identity:create_grant(project_id=5a83c130b5ab41c69c2d74715f3c92d3, user_id=c8df7b14db4b40ae966a31239263eae5, role_id=5a6ded16e6f54de78e356e7b984522fa)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:10.037319 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c79b0d6-30d2-40ce-8f37-cdc273fb40fa None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:10.058074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 145/288] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:13:09 2022] PUT /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3/users/c8df7b14db4b40ae966a31239263eae5/roles/5a6ded16e6f54de78e356e7b984522fa => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:13:10.164112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa0751-bc2a-463d-adbd-a97f424aa58a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:10.190533 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:10.211413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89789f07-d475-4474-8b17-75801118a609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:10.212642 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89789f07-d475-4474-8b17-75801118a609 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:10.213259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89789f07-d475-4474-8b17-75801118a609 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:10.213831 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89789f07-d475-4474-8b17-75801118a609 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:10.228914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89789f07-d475-4474-8b17-75801118a609 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:10.235233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-89789f07-d475-4474-8b17-75801118a609 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:10.235233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:10.235233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:10.235233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:10.235233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:10.235233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:10.235233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:10.235233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:10.235233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:10.235233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:10.235233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:10.235233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:10.235233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:10.235233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:10.235233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:10.235233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:10.236204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:10.236204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:10.236204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:10.236204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:10.236204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:10.236204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:10.236204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:10.236204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:10.236204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:10.236204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:10.236204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:10.236204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:10.236204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:10.236204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:10.236204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:10.236204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:10.236990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:10.236990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:10.236990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:10.236990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:10.236990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:10.236990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:10.236990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:10.236990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:10.236990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:10.236990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:10.236990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:10.236990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:10.240051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 144/289] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:10.247373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79b0d6-30d2-40ce-8f37-cdc273fb40fa None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:10.482342 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c79b0d6-30d2-40ce-8f37-cdc273fb40fa None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:10.565686 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42f1d1-caa0-41e9-893f-8835422648b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:10.566646 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42f1d1-caa0-41e9-893f-8835422648b2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:10.566940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42f1d1-caa0-41e9-893f-8835422648b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:10.567237 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f42f1d1-caa0-41e9-893f-8835422648b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:10.568084 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42f1d1-caa0-41e9-893f-8835422648b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:10.568578 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42f1d1-caa0-41e9-893f-8835422648b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:10.570560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f42f1d1-caa0-41e9-893f-8835422648b2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:10.587652 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 146/290] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:10.592903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89789f07-d475-4474-8b17-75801118a609 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:10.726068 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89789f07-d475-4474-8b17-75801118a609 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:10.762613 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadb5fc-385e-4f40-a9b0-c06a983cccbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:10.763174 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadb5fc-385e-4f40-a9b0-c06a983cccbc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:10.763451 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadb5fc-385e-4f40-a9b0-c06a983cccbc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:10.763792 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aadb5fc-385e-4f40-a9b0-c06a983cccbc None admin] PATH_INFO: `/v3/users/a34d05d5753d400d925d18a235605da8` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:10.764963 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadb5fc-385e-4f40-a9b0-c06a983cccbc None admin] RBAC: Authorizing `identity:get_user(user_id=a34d05d5753d400d925d18a235605da8)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:10.766463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aadb5fc-385e-4f40-a9b0-c06a983cccbc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:10.768224 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 145/291] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:13:10 2022] GET /identity/v3/users/a34d05d5753d400d925d18a235605da8 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:10.774035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42f1d1-caa0-41e9-893f-8835422648b2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:10.794175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:10.910779 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f42f1d1-caa0-41e9-893f-8835422648b2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:10.954742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd8f7f-f522-4573-80c8-34ec63cf0310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:10.955361 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd8f7f-f522-4573-80c8-34ec63cf0310 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:10.955640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd8f7f-f522-4573-80c8-34ec63cf0310 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:10.955912 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd8f7f-f522-4573-80c8-34ec63cf0310 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:11.051968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd8f7f-f522-4573-80c8-34ec63cf0310 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:11.054961 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd8f7f-f522-4573-80c8-34ec63cf0310 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:11.081983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:11.242040 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 147/292] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 15:13:11.247474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadb5fc-385e-4f40-a9b0-c06a983cccbc None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:11.317205 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:11.377250 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aadb5fc-385e-4f40-a9b0-c06a983cccbc None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:11.416997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ce3dc-45f4-4a82-8739-841d06b4f747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:11.417632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ce3dc-45f4-4a82-8739-841d06b4f747 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:11.417913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ce3dc-45f4-4a82-8739-841d06b4f747 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:11.418183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ce3dc-45f4-4a82-8739-841d06b4f747 None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:11.419366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ce3dc-45f4-4a82-8739-841d06b4f747 None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:11.420892 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ce3dc-45f4-4a82-8739-841d06b4f747 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:11.422765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 146/293] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:11 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:11.429276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd8f7f-f522-4573-80c8-34ec63cf0310 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:11.615674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd8f7f-f522-4573-80c8-34ec63cf0310 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:11.643287 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:11.682937 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c9c190-1883-4012-bb74-67078d2b4292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:11.683651 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9c190-1883-4012-bb74-67078d2b4292 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:11.683939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9c190-1883-4012-bb74-67078d2b4292 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:11.684280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c9c190-1883-4012-bb74-67078d2b4292 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:11.685150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9c190-1883-4012-bb74-67078d2b4292 None admin] RBAC: Adding query filter params (role.id=8d794ef8abb94809bd988c5db32c0037, scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3, user.id=a34d05d5753d400d925d18a235605da8) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:11.685456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9c190-1883-4012-bb74-67078d2b4292 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:11.687684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c9c190-1883-4012-bb74-67078d2b4292 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:11.699203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 148/294] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:13:11 2022] GET /identity/v3/role_assignments?user.id=a34d05d5753d400d925d18a235605da8&scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3&role.id=8d794ef8abb94809bd988c5db32c0037 => generated 608 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:11.917433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae75340-1fca-4bb1-bced-b74b2c8d022f None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:11.917872 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae75340-1fca-4bb1-bced-b74b2c8d022f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:11.918048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae75340-1fca-4bb1-bced-b74b2c8d022f None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:11.919837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 147/295] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:11.925392 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c248a7-8cfe-435b-af44-449ccd72343c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:11.925782 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c248a7-8cfe-435b-af44-449ccd72343c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:11.926021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c248a7-8cfe-435b-af44-449ccd72343c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:11.959363 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c9c248a7-8cfe-435b-af44-449ccd72343c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:11.961130 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c9c248a7-8cfe-435b-af44-449ccd72343c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:12.058725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 149/296] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:13:12.359703 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae75340-1fca-4bb1-bced-b74b2c8d022f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:12.531805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae75340-1fca-4bb1-bced-b74b2c8d022f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:12.559741 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:12.565286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6de90c-ce02-48e0-917d-5da0cedc73c8 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:12.565738 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6de90c-ce02-48e0-917d-5da0cedc73c8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:12.566179 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6de90c-ce02-48e0-917d-5da0cedc73c8 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:12.574840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 150/297] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:12 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:13:12.577649 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc3a06-6222-48ee-890d-420d5e9d36fd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:12.578098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc3a06-6222-48ee-890d-420d5e9d36fd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:12.578570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edc3a06-6222-48ee-890d-420d5e9d36fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:12.603624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1c8536-465f-4c7b-9952-fad969071576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:12.604718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c8536-465f-4c7b-9952-fad969071576 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:12.605208 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c8536-465f-4c7b-9952-fad969071576 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:12.605478 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1c8536-465f-4c7b-9952-fad969071576 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:12.607556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7edc3a06-6222-48ee-890d-420d5e9d36fd None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:12.609456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7edc3a06-6222-48ee-890d-420d5e9d36fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:12.619494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1c8536-465f-4c7b-9952-fad969071576 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:12.624853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-8e1c8536-465f-4c7b-9952-fad969071576 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:12.624853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:12.624853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:12.624853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:12.624853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:12.624853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:12.624853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:12.624853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:12.624853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:12.624853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:12.624853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:12.624853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:12.624853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:12.624853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:12.624853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:12.624853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:12.628354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:12.628354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:12.628354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:12.628354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:12.628354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:12.628354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:12.628354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:12.628354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:12.628354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:12.628354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:12.628354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:12.628354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:12.628354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:12.628354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:12.628354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:12.628354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:12.631708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:12.631708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:12.631708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:12.631708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:12.631708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:12.631708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:12.631708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:12.631708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:12.631708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:12.631708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:12.631708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:12.631708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:12.631708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 148/298] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:12.631708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad4b5b-229d-4d94-9762-cea55affb77d None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:12.633049 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad4b5b-229d-4d94-9762-cea55affb77d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:12.633049 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ad4b5b-229d-4d94-9762-cea55affb77d None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:12.633049 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 149/299] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:12.634778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad4b5b-229d-4d94-9762-cea55affb77d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:12.726494 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 151/300] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:13:12.728564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26c3a4-2faf-4c7e-a6a7-5a522a1aaf4c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:12.728948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26c3a4-2faf-4c7e-a6a7-5a522a1aaf4c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:12.729441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26c3a4-2faf-4c7e-a6a7-5a522a1aaf4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:12.755793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2b26c3a4-2faf-4c7e-a6a7-5a522a1aaf4c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:12.757096 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2b26c3a4-2faf-4c7e-a6a7-5a522a1aaf4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:12.802452 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:12.811914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ad4b5b-229d-4d94-9762-cea55affb77d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:12.848143 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 152/301] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:12.864749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c341232-2ffc-491f-ad84-b9a9f729f024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:12.865778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c341232-2ffc-491f-ad84-b9a9f729f024 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:12.866286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c341232-2ffc-491f-ad84-b9a9f729f024 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:12.866768 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c341232-2ffc-491f-ad84-b9a9f729f024 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:12.867684 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c341232-2ffc-491f-ad84-b9a9f729f024 None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:12.868204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c341232-2ffc-491f-ad84-b9a9f729f024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:12.869909 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c341232-2ffc-491f-ad84-b9a9f729f024 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:12.879684 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 150/302] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:12.885392 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26c3a4-2faf-4c7e-a6a7-5a522a1aaf4c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:12.997342 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26c3a4-2faf-4c7e-a6a7-5a522a1aaf4c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:13.008103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c341232-2ffc-491f-ad84-b9a9f729f024 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:13.042850 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6e9f4-a539-4769-b752-34f00ed00774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:13.043555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6e9f4-a539-4769-b752-34f00ed00774 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:13.043833 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6e9f4-a539-4769-b752-34f00ed00774 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:13.044104 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe6e9f4-a539-4769-b752-34f00ed00774 None admin] PATH_INFO: `/v3/users/364a4222d88143d6aade99b011dd76c2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:13.045310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6e9f4-a539-4769-b752-34f00ed00774 None admin] RBAC: Authorizing `identity:get_user(user_id=364a4222d88143d6aade99b011dd76c2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:13.046809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe6e9f4-a539-4769-b752-34f00ed00774 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:13.048423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 153/303] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:13:12 2022] GET /identity/v3/users/364a4222d88143d6aade99b011dd76c2 => generated 293 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:13.054256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6e9f4-a539-4769-b752-34f00ed00774 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:13.059548 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:13.102033 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:13.157855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c341232-2ffc-491f-ad84-b9a9f729f024 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:13.179046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe6e9f4-a539-4769-b752-34f00ed00774 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:13.194875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d12300-e28a-4b66-80dc-cc37ce816c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:13.195594 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d12300-e28a-4b66-80dc-cc37ce816c8c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:13.195874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d12300-e28a-4b66-80dc-cc37ce816c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:13.196143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d12300-e28a-4b66-80dc-cc37ce816c8c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:13.206945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d12300-e28a-4b66-80dc-cc37ce816c8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:13.213185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-98d12300-e28a-4b66-80dc-cc37ce816c8c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:13:13.213185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:13.213185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:13.213185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:13.213185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:13.213185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:13.213185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:13.213185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:13.213185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:13.213185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:13.213185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:13.213185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:13.213185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:13.213185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:13.213185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:13.213185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:13.214318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:13.214318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:13.214318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:13.214318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:13.214318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:13.214318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:13.214318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:13.214318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:13.214318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:13.214318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:13.214318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:13.214318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:13.214318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:13.214318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:13.214318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:13.214318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:13.215125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:13.215125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:13.215125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:13.215125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:13.215125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:13.215125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:13.215125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:13.215125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:13.215125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:13.215125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:13.215125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:13:13.215125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:13.215125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 151/304] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:13:13.215125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb591c-1574-4f82-b235-8158d26b08ab None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:13.215815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb591c-1574-4f82-b235-8158d26b08ab None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:13.215815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb591c-1574-4f82-b235-8158d26b08ab None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:13.221350 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 152/305] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:13.221350 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb591c-1574-4f82-b235-8158d26b08ab None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:13.233327 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3dec0b-b2df-449d-9cb0-8b2d811fcd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:13.234346 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dec0b-b2df-449d-9cb0-8b2d811fcd0a None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:13.234712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dec0b-b2df-449d-9cb0-8b2d811fcd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:13.234994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3dec0b-b2df-449d-9cb0-8b2d811fcd0a None admin] PATH_INFO: `/v3/system/users/364a4222d88143d6aade99b011dd76c2/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:13.236716 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3dec0b-b2df-449d-9cb0-8b2d811fcd0a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=364a4222d88143d6aade99b011dd76c2, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:13.238092 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3dec0b-b2df-449d-9cb0-8b2d811fcd0a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:13.248147 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 154/306] 10.222.0.76 () {58 vars in 1334 bytes} [Wed Aug 24 15:13:13 2022] PUT /identity/v3/system/users/364a4222d88143d6aade99b011dd76c2/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:13:13.262526 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3dec0b-b2df-449d-9cb0-8b2d811fcd0a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:13.351890 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:13.359361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb591c-1574-4f82-b235-8158d26b08ab None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:13.396215 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:13.400224 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3dec0b-b2df-449d-9cb0-8b2d811fcd0a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:13.408692 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996267dd-07a9-4f19-b7b1-a3d7e0b3add5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:13.410640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996267dd-07a9-4f19-b7b1-a3d7e0b3add5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:13.411090 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996267dd-07a9-4f19-b7b1-a3d7e0b3add5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:13.411533 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996267dd-07a9-4f19-b7b1-a3d7e0b3add5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:13.414993 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996267dd-07a9-4f19-b7b1-a3d7e0b3add5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:13.417606 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996267dd-07a9-4f19-b7b1-a3d7e0b3add5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:13.420430 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 153/307] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:13.421872 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c46a41-33b9-4dfe-bb76-516151706667 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:13.422159 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c46a41-33b9-4dfe-bb76-516151706667 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:13.422453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c46a41-33b9-4dfe-bb76-516151706667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:13.442168 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d3c46a41-33b9-4dfe-bb76-516151706667 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:13.443392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d3c46a41-33b9-4dfe-bb76-516151706667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:13.446420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82517061-fa02-487a-aa67-88445d36d7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:13.447374 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82517061-fa02-487a-aa67-88445d36d7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:13.447836 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82517061-fa02-487a-aa67-88445d36d7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:13.448313 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82517061-fa02-487a-aa67-88445d36d7f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:13.449141 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82517061-fa02-487a-aa67-88445d36d7f8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:13.449583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82517061-fa02-487a-aa67-88445d36d7f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:13.451153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82517061-fa02-487a-aa67-88445d36d7f8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:13.463489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 155/308] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:13.464266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82517061-fa02-487a-aa67-88445d36d7f8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:13.526206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 154/309] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:13.536099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c46a41-33b9-4dfe-bb76-516151706667 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:13.599839 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82517061-fa02-487a-aa67-88445d36d7f8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:13.647870 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c6fe9-8030-44b9-8b9e-8978c011c4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:13.648534 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c6fe9-8030-44b9-8b9e-8978c011c4ca None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:13.648820 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c6fe9-8030-44b9-8b9e-8978c011c4ca None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:13.649099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6c6fe9-8030-44b9-8b9e-8978c011c4ca None admin] PATH_INFO: `/v3/groups` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:13.650042 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c6fe9-8030-44b9-8b9e-8978c011c4ca None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:13.651798 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6c6fe9-8030-44b9-8b9e-8978c011c4ca None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:13.658364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:13.668947 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c46a41-33b9-4dfe-bb76-516151706667 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:13.673868 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 156/310] 10.222.0.76 () {60 vars in 1134 bytes} [Wed Aug 24 15:13:13 2022] POST /identity/v3/groups => generated 229 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:13:13.717163 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb285e-4fae-4f51-b16d-6883d9cba3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:13.717815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb285e-4fae-4f51-b16d-6883d9cba3c5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:13.718082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb285e-4fae-4f51-b16d-6883d9cba3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:13.718371 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeb285e-4fae-4f51-b16d-6883d9cba3c5 None admin] PATH_INFO: `/v3/users/5f24fabc2ab74f8b87c7c9e0df49ae00` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:13.737284 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb285e-4fae-4f51-b16d-6883d9cba3c5 None admin] RBAC: Authorizing `identity:get_user(user_id=5f24fabc2ab74f8b87c7c9e0df49ae00)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:13.738819 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aeb285e-4fae-4f51-b16d-6883d9cba3c5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:13.741082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 155/311] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:13:13 2022] GET /identity/v3/users/5f24fabc2ab74f8b87c7c9e0df49ae00 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:13.747585 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c6fe9-8030-44b9-8b9e-8978c011c4ca None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:13.779042 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:13.844128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb285e-4fae-4f51-b16d-6883d9cba3c5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:13.887111 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6c6fe9-8030-44b9-8b9e-8978c011c4ca None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:13.940930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468eedc1-0e91-446f-9755-4f1bdd165e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:13.941790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eedc1-0e91-446f-9755-4f1bdd165e57 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:13.942304 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eedc1-0e91-446f-9755-4f1bdd165e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:13.942846 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468eedc1-0e91-446f-9755-4f1bdd165e57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:13.943599 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468eedc1-0e91-446f-9755-4f1bdd165e57 None admin] RBAC: Adding query filter params (role.id=5a6ded16e6f54de78e356e7b984522fa, user.id=5f24fabc2ab74f8b87c7c9e0df49ae00, scope.system=all) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:13.943940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468eedc1-0e91-446f-9755-4f1bdd165e57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:13.945401 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468eedc1-0e91-446f-9755-4f1bdd165e57 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:13.951920 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 157/312] 10.222.0.76 () {56 vars in 1308 bytes} [Wed Aug 24 15:13:13 2022] GET /identity/v3/role_assignments?user.id=5f24fabc2ab74f8b87c7c9e0df49ae00&scope.system=all&role.id=5a6ded16e6f54de78e356e7b984522fa => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:13.999060 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeb285e-4fae-4f51-b16d-6883d9cba3c5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:14.039419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ada1b6-0317-4ea8-a5b6-9ab07a950f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:14.040069 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ada1b6-0317-4ea8-a5b6-9ab07a950f4e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:14.040345 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ada1b6-0317-4ea8-a5b6-9ab07a950f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:14.040614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ada1b6-0317-4ea8-a5b6-9ab07a950f4e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:14.053123 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ada1b6-0317-4ea8-a5b6-9ab07a950f4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:14.058803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-26ada1b6-0317-4ea8-a5b6-9ab07a950f4e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:13:14.058803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:14.058803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:14.058803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:14.058803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:14.058803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:14.058803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:14.058803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:14.058803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:14.058803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:14.058803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:14.058803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:14.058803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:14.058803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:14.058803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:14.058803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:14.059724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:14.059724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:14.059724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:14.059724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:14.059724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:14.059724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:14.059724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:14.059724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:14.059724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:14.059724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:14.059724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:14.059724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:14.059724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:14.059724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:14.059724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:14.059724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:14.061462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:14.061462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:14.061462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:14.061462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:14.061462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:14.061462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:14.061462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:14.061462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:14.061462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:14.061462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:14.061462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:13:14.061462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:14.061462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 156/313] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:14.068572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468eedc1-0e91-446f-9755-4f1bdd165e57 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:14.192049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:14.212700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468eedc1-0e91-446f-9755-4f1bdd165e57 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:14.262168 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb02a16-df04-4760-803d-a9ad770d10ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:14.263149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb02a16-df04-4760-803d-a9ad770d10ca None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:14.263598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb02a16-df04-4760-803d-a9ad770d10ca None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:14.264046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb02a16-df04-4760-803d-a9ad770d10ca None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:14.265131 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb02a16-df04-4760-803d-a9ad770d10ca None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:14.265548 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb02a16-df04-4760-803d-a9ad770d10ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:14.267013 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb02a16-df04-4760-803d-a9ad770d10ca None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:14.276731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 158/314] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:14.282732 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ada1b6-0317-4ea8-a5b6-9ab07a950f4e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:14.317388 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:14.405814 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ada1b6-0317-4ea8-a5b6-9ab07a950f4e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:14.447004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30232-5326-44df-96fb-583d2442a7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:14.447627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30232-5326-44df-96fb-583d2442a7f5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:14.447862 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30232-5326-44df-96fb-583d2442a7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:14.448069 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30232-5326-44df-96fb-583d2442a7f5 None admin] PATH_INFO: `/v3/users/c8df7b14db4b40ae966a31239263eae5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:14.449192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe30232-5326-44df-96fb-583d2442a7f5 None admin] RBAC: Authorizing `identity:get_user(user_id=c8df7b14db4b40ae966a31239263eae5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:14.450737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe30232-5326-44df-96fb-583d2442a7f5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:14.452906 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 157/315] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:13:14 2022] GET /identity/v3/users/c8df7b14db4b40ae966a31239263eae5 => generated 297 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:14.458398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb02a16-df04-4760-803d-a9ad770d10ca None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:14.583009 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb02a16-df04-4760-803d-a9ad770d10ca None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:14.618015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a96f10-cc5e-4e91-81b9-26ef537acbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:14.618660 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a96f10-cc5e-4e91-81b9-26ef537acbe3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:14.618941 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a96f10-cc5e-4e91-81b9-26ef537acbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:14.619215 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a96f10-cc5e-4e91-81b9-26ef537acbe3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:14.642302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:14.697198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a96f10-cc5e-4e91-81b9-26ef537acbe3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:14.698585 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a96f10-cc5e-4e91-81b9-26ef537acbe3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:14.802050 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 159/316] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:14.808527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30232-5326-44df-96fb-583d2442a7f5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:14.893486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:14.913544 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30232-5326-44df-96fb-583d2442a7f5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:14.956442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a17377-3add-4170-9fb5-a66141505e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:14.957423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a17377-3add-4170-9fb5-a66141505e6c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:14.957908 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a17377-3add-4170-9fb5-a66141505e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:14.958411 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a17377-3add-4170-9fb5-a66141505e6c None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:14.960422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a17377-3add-4170-9fb5-a66141505e6c None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:14.962137 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a17377-3add-4170-9fb5-a66141505e6c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:14.964006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 158/317] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:14 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:14.974338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a96f10-cc5e-4e91-81b9-26ef537acbe3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:15.022738 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:15.078402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a96f10-cc5e-4e91-81b9-26ef537acbe3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:15.114842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd3dc1-cb2c-4f73-80e0-a258c29f5ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:15.115475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd3dc1-cb2c-4f73-80e0-a258c29f5ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:15.116267 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd3dc1-cb2c-4f73-80e0-a258c29f5ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:15.116541 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd3dc1-cb2c-4f73-80e0-a258c29f5ef2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:15.117278 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd3dc1-cb2c-4f73-80e0-a258c29f5ef2 None admin] RBAC: Adding query filter params (role.id=5a6ded16e6f54de78e356e7b984522fa, scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3, user.id=c8df7b14db4b40ae966a31239263eae5) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:15.117583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd3dc1-cb2c-4f73-80e0-a258c29f5ef2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:15.119034 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd3dc1-cb2c-4f73-80e0-a258c29f5ef2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:15.130366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 160/318] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:13:14 2022] GET /identity/v3/role_assignments?user.id=c8df7b14db4b40ae966a31239263eae5&scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3&role.id=5a6ded16e6f54de78e356e7b984522fa => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:15.303854 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349cb6e-5510-4af8-8698-bb3b6f86b4e1 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:15.304059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349cb6e-5510-4af8-8698-bb3b6f86b4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:15.304337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5349cb6e-5510-4af8-8698-bb3b6f86b4e1 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:15.305830 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 159/319] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:15.311675 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c380be-0058-4dc4-b1cd-59547d721025 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:15.312251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c380be-0058-4dc4-b1cd-59547d721025 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:15.312718 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c380be-0058-4dc4-b1cd-59547d721025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:15.332295 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b8c380be-0058-4dc4-b1cd-59547d721025 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:15.333816 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b8c380be-0058-4dc4-b1cd-59547d721025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:15.413605 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 161/320] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:15.682510 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5349cb6e-5510-4af8-8698-bb3b6f86b4e1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:15.792307 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5349cb6e-5510-4af8-8698-bb3b6f86b4e1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:15.795380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32455dab-1606-47fd-86c3-fa71c42b159d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:15.795664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32455dab-1606-47fd-86c3-fa71c42b159d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:15.795937 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32455dab-1606-47fd-86c3-fa71c42b159d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:15.797437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 162/321] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:13:15.802445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9b5eb-21f8-45d8-bfcf-8080f994cb9e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:15.803901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9b5eb-21f8-45d8-bfcf-8080f994cb9e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:15.803901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad9b5eb-21f8-45d8-bfcf-8080f994cb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:15.821922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cad9b5eb-21f8-45d8-bfcf-8080f994cb9e None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:15.823165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cad9b5eb-21f8-45d8-bfcf-8080f994cb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:15.830840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15d610-a9a4-41aa-a7cd-c075d3a76e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:15.831581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15d610-a9a4-41aa-a7cd-c075d3a76e09 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:15.831745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15d610-a9a4-41aa-a7cd-c075d3a76e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:15.831993 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15d610-a9a4-41aa-a7cd-c075d3a76e09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:15.846494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15d610-a9a4-41aa-a7cd-c075d3a76e09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:15.852953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-5a15d610-a9a4-41aa-a7cd-c075d3a76e09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:15.852953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:15.852953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:15.852953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:15.852953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:15.852953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:15.852953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:15.852953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:15.852953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:15.852953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:15.852953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:15.852953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:15.852953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:15.852953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:15.852953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:15.852953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:15.855282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:15.855282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:15.855282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:15.855282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:15.855282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:15.855282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:15.855282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:15.855282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:15.855282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:15.855282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:15.855282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:15.855282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:15.855282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:15.855282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:15.855282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:15.855282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:15.856751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:15.856751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:15.856751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:15.856751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:15.856751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:15.856751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:15.856751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:15.856751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:15.856751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:15.856751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:15.856751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:15.856751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:15.856751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 160/322] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:13:15.859603 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15d610-a9a4-41aa-a7cd-c075d3a76e09 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:15.873005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:15.874929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:15.908186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 163/323] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:13:15.978270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15d610-a9a4-41aa-a7cd-c075d3a76e09 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:16.016617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955cd1ca-b715-48c7-ae6f-4918f30fd319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:16.017230 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cd1ca-b715-48c7-ae6f-4918f30fd319 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:16.017508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cd1ca-b715-48c7-ae6f-4918f30fd319 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:16.017779 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955cd1ca-b715-48c7-ae6f-4918f30fd319 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:16.018468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955cd1ca-b715-48c7-ae6f-4918f30fd319 None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:16.018771 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955cd1ca-b715-48c7-ae6f-4918f30fd319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:16.020140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955cd1ca-b715-48c7-ae6f-4918f30fd319 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:16.029820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 161/324] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:16.035339 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9b5eb-21f8-45d8-bfcf-8080f994cb9e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:16.064279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7c2ff-8336-42ae-b8c5-b0fe1121bab4 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:16.064563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7c2ff-8336-42ae-b8c5-b0fe1121bab4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:16.064834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be7c2ff-8336-42ae-b8c5-b0fe1121bab4 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:16.066267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 162/325] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:13:16.071377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7e6eb-5ada-42c9-a0a4-ebab4b126186 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:16.071652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7e6eb-5ada-42c9-a0a4-ebab4b126186 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:16.071922 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7e6eb-5ada-42c9-a0a4-ebab4b126186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:16.090522 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-19e7e6eb-5ada-42c9-a0a4-ebab4b126186 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:16.091726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-19e7e6eb-5ada-42c9-a0a4-ebab4b126186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:16.145429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad9b5eb-21f8-45d8-bfcf-8080f994cb9e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:16.162727 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:16.167929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 163/326] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:13:16.184022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7e6eb-5ada-42c9-a0a4-ebab4b126186 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:16.184580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540699db-8380-48d5-8b03-5d06c8b036cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:16.185210 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540699db-8380-48d5-8b03-5d06c8b036cd None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:16.185663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540699db-8380-48d5-8b03-5d06c8b036cd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:16.185936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540699db-8380-48d5-8b03-5d06c8b036cd None admin] PATH_INFO: `/v3/users/364a4222d88143d6aade99b011dd76c2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:16.187354 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540699db-8380-48d5-8b03-5d06c8b036cd None admin] RBAC: Authorizing `identity:get_user(user_id=364a4222d88143d6aade99b011dd76c2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:16.188794 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540699db-8380-48d5-8b03-5d06c8b036cd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:16.194800 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 164/327] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:13:16 2022] GET /identity/v3/users/364a4222d88143d6aade99b011dd76c2 => generated 293 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:16.199292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540699db-8380-48d5-8b03-5d06c8b036cd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:16.255777 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:16.323958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7e6eb-5ada-42c9-a0a4-ebab4b126186 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:16.333272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540699db-8380-48d5-8b03-5d06c8b036cd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:16.360172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014e161-6d22-4b8d-8b58-25bfc7437ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:16.360810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014e161-6d22-4b8d-8b58-25bfc7437ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:16.361084 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014e161-6d22-4b8d-8b58-25bfc7437ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:16.361360 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014e161-6d22-4b8d-8b58-25bfc7437ae7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:16.362652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014e161-6d22-4b8d-8b58-25bfc7437ae7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:16.364139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014e161-6d22-4b8d-8b58-25bfc7437ae7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:16.365941 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 164/328] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:16.371157 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014e161-6d22-4b8d-8b58-25bfc7437ae7 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:16.372056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a4886-0aed-4203-a7f8-a8ff23d17745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:16.372666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a4886-0aed-4203-a7f8-a8ff23d17745 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:16.372945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a4886-0aed-4203-a7f8-a8ff23d17745 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:16.373215 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a4886-0aed-4203-a7f8-a8ff23d17745 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:16.374069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a4886-0aed-4203-a7f8-a8ff23d17745 None admin] RBAC: Adding query filter params (role.id=8d794ef8abb94809bd988c5db32c0037, user.id=364a4222d88143d6aade99b011dd76c2, scope.system=all) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:16.374512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a4886-0aed-4203-a7f8-a8ff23d17745 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:16.375944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a4886-0aed-4203-a7f8-a8ff23d17745 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:16.388805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 165/329] 10.222.0.76 () {56 vars in 1308 bytes} [Wed Aug 24 15:13:16 2022] GET /identity/v3/role_assignments?user.id=364a4222d88143d6aade99b011dd76c2&scope.system=all&role.id=8d794ef8abb94809bd988c5db32c0037 => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:16.437575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a4886-0aed-4203-a7f8-a8ff23d17745 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:16.449212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:16.492977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014e161-6d22-4b8d-8b58-25bfc7437ae7 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:16.547096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363a680-c467-4c46-86c6-21627ee9c1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:16.547810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363a680-c467-4c46-86c6-21627ee9c1a8 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:16.548163 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363a680-c467-4c46-86c6-21627ee9c1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:16.548441 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7363a680-c467-4c46-86c6-21627ee9c1a8 None admin] PATH_INFO: `/v3/groups` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:16.549067 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363a680-c467-4c46-86c6-21627ee9c1a8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:16.550538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7363a680-c467-4c46-86c6-21627ee9c1a8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:16.569965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:16.574247 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 165/330] 10.222.0.76 () {60 vars in 1134 bytes} [Wed Aug 24 15:13:16 2022] POST /identity/v3/groups => generated 226 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:13:16.574860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a4886-0aed-4203-a7f8-a8ff23d17745 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:16.620731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e99fd-5586-4ccf-a6b7-0c4ebc77bcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:16.621432 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e99fd-5586-4ccf-a6b7-0c4ebc77bcbd None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:16.621688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e99fd-5586-4ccf-a6b7-0c4ebc77bcbd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:16.622019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e99fd-5586-4ccf-a6b7-0c4ebc77bcbd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:16.634718 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e99fd-5586-4ccf-a6b7-0c4ebc77bcbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:16.640272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c05e99fd-5586-4ccf-a6b7-0c4ebc77bcbd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:13:16.640272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:16.640272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:16.640272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:16.640272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:16.640272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:16.640272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:16.640272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:16.640272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:16.640272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:16.640272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:16.640272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:16.640272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:16.640272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:16.640272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:16.640272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:16.641430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:16.641430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:16.641430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:16.641430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:16.641430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:16.641430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:16.641430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:16.641430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:16.641430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:16.641430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:16.641430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:16.641430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:16.641430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:16.641430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:16.641430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:16.641430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:16.642484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:16.642484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:16.642484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:16.642484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:16.642484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:16.642484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:16.642484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:16.642484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:16.642484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:16.642484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:16.642484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:13:16.642484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:16.642484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 166/331] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:13:16.648471 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363a680-c467-4c46-86c6-21627ee9c1a8 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:16.759845 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7363a680-c467-4c46-86c6-21627ee9c1a8 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:16.786050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:16.794960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d505720-b0cf-499f-8376-52e2965ef1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:16.795631 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d505720-b0cf-499f-8376-52e2965ef1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:16.796313 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d505720-b0cf-499f-8376-52e2965ef1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:16.796593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d505720-b0cf-499f-8376-52e2965ef1f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:16.797258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d505720-b0cf-499f-8376-52e2965ef1f5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:16.797556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d505720-b0cf-499f-8376-52e2965ef1f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:16.798952 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d505720-b0cf-499f-8376-52e2965ef1f5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:16.807811 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 166/332] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:16.813380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e99fd-5586-4ccf-a6b7-0c4ebc77bcbd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:16.924362 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e99fd-5586-4ccf-a6b7-0c4ebc77bcbd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:16.961337 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe6351a-b1db-42d5-8992-706baea6db46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:16.962063 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6351a-b1db-42d5-8992-706baea6db46 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:16.962506 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6351a-b1db-42d5-8992-706baea6db46 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:16.962787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe6351a-b1db-42d5-8992-706baea6db46 None admin] PATH_INFO: `/v3/users/5f24fabc2ab74f8b87c7c9e0df49ae00` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:16.963947 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6351a-b1db-42d5-8992-706baea6db46 None admin] RBAC: Authorizing `identity:get_user(user_id=5f24fabc2ab74f8b87c7c9e0df49ae00)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:16.965400 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe6351a-b1db-42d5-8992-706baea6db46 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:16.967444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 167/333] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:13:16 2022] GET /identity/v3/users/5f24fabc2ab74f8b87c7c9e0df49ae00 => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:16.973687 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d505720-b0cf-499f-8376-52e2965ef1f5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:17.083780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d505720-b0cf-499f-8376-52e2965ef1f5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:17.119713 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9a00fd-9ae5-4600-b9ef-704db4efaae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:17.120372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a00fd-9ae5-4600-b9ef-704db4efaae8 None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:17.120650 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a00fd-9ae5-4600-b9ef-704db4efaae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:17.120920 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9a00fd-9ae5-4600-b9ef-704db4efaae8 None admin] PATH_INFO: `/v3/system/users/5f24fabc2ab74f8b87c7c9e0df49ae00/roles/5a6ded16e6f54de78e356e7b984522fa` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:17.122686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a00fd-9ae5-4600-b9ef-704db4efaae8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=5f24fabc2ab74f8b87c7c9e0df49ae00, role_id=5a6ded16e6f54de78e356e7b984522fa)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:17.124182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9a00fd-9ae5-4600-b9ef-704db4efaae8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:17.134375 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 167/334] 10.222.0.76 () {58 vars in 1334 bytes} [Wed Aug 24 15:13:16 2022] PUT /identity/v3/system/users/5f24fabc2ab74f8b87c7c9e0df49ae00/roles/5a6ded16e6f54de78e356e7b984522fa => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:13:19.164201 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b0e6d-e236-4694-8a9a-e2c1d534d895 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:19.164714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b0e6d-e236-4694-8a9a-e2c1d534d895 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:19.164714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187b0e6d-e236-4694-8a9a-e2c1d534d895 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:19.166068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 168/335] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:19.171754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459d3c0-76b3-46e7-af05-ae0b1adc6fa2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:19.172216 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459d3c0-76b3-46e7-af05-ae0b1adc6fa2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:19.172291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459d3c0-76b3-46e7-af05-ae0b1adc6fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:19.198896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4459d3c0-76b3-46e7-af05-ae0b1adc6fa2 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:19.200102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4459d3c0-76b3-46e7-af05-ae0b1adc6fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:19.213287 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49eed8f-6e34-4494-9126-9f1877b613d7 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:19.213905 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49eed8f-6e34-4494-9126-9f1877b613d7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:19.214407 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49eed8f-6e34-4494-9126-9f1877b613d7 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:19.216174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 169/336] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:19.221369 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841128d-c07c-4cc0-89c0-c674b9b54b52 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:19.221906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841128d-c07c-4cc0-89c0-c674b9b54b52 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:19.222422 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a841128d-c07c-4cc0-89c0-c674b9b54b52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:19.251237 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a841128d-c07c-4cc0-89c0-c674b9b54b52 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:19.252757 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a841128d-c07c-4cc0-89c0-c674b9b54b52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:19.264324 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:19.286268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:19.293373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 168/337] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:19.294665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4227d-14ff-4889-a165-e25ec6b8ae38 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:19.294950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4227d-14ff-4889-a165-e25ec6b8ae38 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:19.295220 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4227d-14ff-4889-a165-e25ec6b8ae38 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:19.296447 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 169/338] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:19.304238 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48821a88-b6cd-44eb-8265-af05abaf7812 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:19.304517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48821a88-b6cd-44eb-8265-af05abaf7812 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:19.304787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48821a88-b6cd-44eb-8265-af05abaf7812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:19.338301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-48821a88-b6cd-44eb-8265-af05abaf7812 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:19.339546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-48821a88-b6cd-44eb-8265-af05abaf7812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:19.406341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 170/339] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:19.452532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 170/340] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:13:19.578286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d55ff4-ea9e-4c11-9c04-eddff627e91b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:19.578602 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d55ff4-ea9e-4c11-9c04-eddff627e91b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:19.578832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d55ff4-ea9e-4c11-9c04-eddff627e91b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:19.580427 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 171/341] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:19.585873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f39b50-ad39-4ddb-9237-dcda85cf7af0 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:19.586163 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f39b50-ad39-4ddb-9237-dcda85cf7af0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:19.586456 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f39b50-ad39-4ddb-9237-dcda85cf7af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:19.615193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c8f39b50-ad39-4ddb-9237-dcda85cf7af0 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:19.616928 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c8f39b50-ad39-4ddb-9237-dcda85cf7af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:19.704859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 171/342] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:19.817515 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d55ff4-ea9e-4c11-9c04-eddff627e91b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:19.947832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d55ff4-ea9e-4c11-9c04-eddff627e91b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:19.996338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc16e98-e2ee-49bf-9486-7296fc6b798d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:19.997319 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc16e98-e2ee-49bf-9486-7296fc6b798d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:19.997804 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc16e98-e2ee-49bf-9486-7296fc6b798d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:19.998287 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc16e98-e2ee-49bf-9486-7296fc6b798d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:20.011571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f39b50-ad39-4ddb-9237-dcda85cf7af0 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:20.015548 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc16e98-e2ee-49bf-9486-7296fc6b798d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:20.021416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-bcc16e98-e2ee-49bf-9486-7296fc6b798d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:20.021416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:20.021416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:20.021416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:20.021416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:20.021416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:20.021416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:20.021416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:20.021416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:20.021416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:20.021416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:20.021416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:20.021416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:20.021416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:20.021416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:20.021416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:20.022958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:20.022958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:20.022958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:20.022958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:20.022958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:20.022958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:20.022958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:20.022958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:20.022958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:20.022958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:20.022958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:20.022958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:20.022958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:20.022958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:20.022958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:20.022958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:20.023735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:20.023735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:20.023735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:20.023735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:20.023735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:20.023735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:20.023735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:20.023735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:20.023735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:20.023735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:20.023735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:20.023735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:20.026283 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 172/343] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:20.027348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc16e98-e2ee-49bf-9486-7296fc6b798d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:20.062906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:20.088044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:20.163491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f39b50-ad39-4ddb-9237-dcda85cf7af0 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:20.178348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc16e98-e2ee-49bf-9486-7296fc6b798d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:20.210669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d04962-498c-42d2-b99a-dd76cc01e55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:20.211982 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d04962-498c-42d2-b99a-dd76cc01e55f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:20.212395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d04962-498c-42d2-b99a-dd76cc01e55f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:20.212914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d04962-498c-42d2-b99a-dd76cc01e55f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:20.221558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b122b6-6f14-4583-ba7f-db5740d29f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:20.222434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b122b6-6f14-4583-ba7f-db5740d29f66 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:20.222927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b122b6-6f14-4583-ba7f-db5740d29f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:20.223387 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b122b6-6f14-4583-ba7f-db5740d29f66 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:20.229827 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d04962-498c-42d2-b99a-dd76cc01e55f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:20.236241 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b122b6-6f14-4583-ba7f-db5740d29f66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:20.237140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-85d04962-498c-42d2-b99a-dd76cc01e55f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:20.237140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:20.237140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:20.237140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:20.237140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:20.237140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:20.237140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:20.237140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:20.237140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:20.237140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:20.237140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:20.237140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:20.237140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:20.237140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:20.237140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:20.237140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:20.238089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:20.238089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:20.238089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:20.238089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:20.238089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:20.238089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:20.238089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:20.238089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:20.238089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:20.238089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:20.238089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:20.238089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:20.238089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:20.238089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:20.238089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:20.238089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:20.238911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:20.238911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:20.238911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:20.238911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:20.238911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:20.238911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:20.238911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:20.238911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:20.238911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:20.238911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:20.238911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:20.238911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:20.241690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 172/344] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:20.242725 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d04962-498c-42d2-b99a-dd76cc01e55f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:20.243545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e2b122b6-6f14-4583-ba7f-db5740d29f66 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:20.243545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:20.243545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:20.243545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:20.243545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:20.243545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:20.243545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:20.243545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:20.243545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:20.243545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:20.243545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:20.243545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:20.243545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:20.243545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:20.243545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:20.243545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:20.244420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:20.244420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:20.244420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:20.244420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:20.244420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:20.244420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:20.244420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:20.244420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:20.244420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:20.244420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:20.244420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:20.244420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:20.244420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:20.244420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:20.244420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:20.244420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:20.245247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:20.245247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:20.245247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:20.245247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:20.245247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:20.245247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:20.245247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:20.245247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:20.245247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:20.245247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:20.245247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:20.245247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:20.245247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 173/345] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:13:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:20.245895 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b122b6-6f14-4583-ba7f-db5740d29f66 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:20.339723 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:20.370871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b122b6-6f14-4583-ba7f-db5740d29f66 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:20.373888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d04962-498c-42d2-b99a-dd76cc01e55f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:20.377813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:20.415383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e53ef-4bf0-4033-b433-3c45ff38712a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:20.416607 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e53ef-4bf0-4033-b433-3c45ff38712a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:20.417076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e53ef-4bf0-4033-b433-3c45ff38712a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:20.417534 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e53ef-4bf0-4033-b433-3c45ff38712a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:20.418520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30feb99-6b19-443b-a146-e67c22e66d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:20.419138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30feb99-6b19-443b-a146-e67c22e66d8e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:20.420583 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30feb99-6b19-443b-a146-e67c22e66d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:20.420763 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30feb99-6b19-443b-a146-e67c22e66d8e None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:20.421442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30feb99-6b19-443b-a146-e67c22e66d8e None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:20.421747 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30feb99-6b19-443b-a146-e67c22e66d8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:20.423549 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30feb99-6b19-443b-a146-e67c22e66d8e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:20.431621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e53ef-4bf0-4033-b433-3c45ff38712a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:20.434264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 173/346] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:20.435102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30feb99-6b19-443b-a146-e67c22e66d8e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:20.437948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-cc6e53ef-4bf0-4033-b433-3c45ff38712a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:13:20.437948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:20.437948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:20.437948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:20.437948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:20.437948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:20.437948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:20.437948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:20.437948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:20.437948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:20.437948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:20.437948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:20.437948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:20.437948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:20.437948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:20.437948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:20.438842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:20.438842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:20.438842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:20.438842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:20.438842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:20.438842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:20.438842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:20.438842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:20.438842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:20.438842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:20.438842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:20.438842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:20.438842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:20.438842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:20.438842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:20.438842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:20.439617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:20.439617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:20.439617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:20.439617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:20.439617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:20.439617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:20.439617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:20.439617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:20.439617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:20.439617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:20.439617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 15:13:20.439617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:20.442440 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 174/347] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:20.443495 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e53ef-4bf0-4033-b433-3c45ff38712a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:20.563081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30feb99-6b19-443b-a146-e67c22e66d8e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:20.579364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e53ef-4bf0-4033-b433-3c45ff38712a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:20.612765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6c5a0-e82f-4de7-83c7-3a6febfb8bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:20.613295 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6c5a0-e82f-4de7-83c7-3a6febfb8bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:20.613602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6c5a0-e82f-4de7-83c7-3a6febfb8bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:20.613837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6c5a0-e82f-4de7-83c7-3a6febfb8bb4 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:20.614560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6c5a0-e82f-4de7-83c7-3a6febfb8bb4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:20.614859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6c5a0-e82f-4de7-83c7-3a6febfb8bb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:20.616521 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f6c5a0-e82f-4de7-83c7-3a6febfb8bb4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:20.621477 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdc538-1427-4c88-bbe4-396a315f6e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:20.622091 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdc538-1427-4c88-bbe4-396a315f6e8d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:20.622402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdc538-1427-4c88-bbe4-396a315f6e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:20.622674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fdc538-1427-4c88-bbe4-396a315f6e8d None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:20.623340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdc538-1427-4c88-bbe4-396a315f6e8d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:20.623644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdc538-1427-4c88-bbe4-396a315f6e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:20.625022 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fdc538-1427-4c88-bbe4-396a315f6e8d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:20.626359 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 174/348] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:20.627136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6c5a0-e82f-4de7-83c7-3a6febfb8bb4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:20.631962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:20.642289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 175/349] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:20.642289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdc538-1427-4c88-bbe4-396a315f6e8d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:20.668897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:20.774419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6c5a0-e82f-4de7-83c7-3a6febfb8bb4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:20.788793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fdc538-1427-4c88-bbe4-396a315f6e8d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:20.819826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699d3bf-3789-4d1e-8bf6-d5cb36837797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:20.820663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699d3bf-3789-4d1e-8bf6-d5cb36837797 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:20.821013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699d3bf-3789-4d1e-8bf6-d5cb36837797 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:20.821355 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9699d3bf-3789-4d1e-8bf6-d5cb36837797 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:20.838795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf16b48-bb86-40be-8e30-89446450b751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:20.839961 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf16b48-bb86-40be-8e30-89446450b751 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:20.839961 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf16b48-bb86-40be-8e30-89446450b751 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:20.839961 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf16b48-bb86-40be-8e30-89446450b751 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:20.840599 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf16b48-bb86-40be-8e30-89446450b751 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:20.840898 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf16b48-bb86-40be-8e30-89446450b751 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:20.842594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf16b48-bb86-40be-8e30-89446450b751 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:20.852788 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 176/350] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:20.855841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38904c-d686-42d9-9109-ce2a77a8e452 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:20.856294 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38904c-d686-42d9-9109-ce2a77a8e452 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:20.856741 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38904c-d686-42d9-9109-ce2a77a8e452 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:20.858643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 177/351] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:20.859564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38904c-d686-42d9-9109-ce2a77a8e452 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:20.922773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699d3bf-3789-4d1e-8bf6-d5cb36837797 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:20.924266 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9699d3bf-3789-4d1e-8bf6-d5cb36837797 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:20.942478 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:20.961200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:21.014002 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38904c-d686-42d9-9109-ce2a77a8e452 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:21.058165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f672c1-4afd-4ba5-b4db-d64773638f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:21.058798 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f672c1-4afd-4ba5-b4db-d64773638f70 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:21.059080 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f672c1-4afd-4ba5-b4db-d64773638f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:21.059346 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f672c1-4afd-4ba5-b4db-d64773638f70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:21.063287 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 175/352] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:21.064043 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699d3bf-3789-4d1e-8bf6-d5cb36837797 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:21.141716 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f672c1-4afd-4ba5-b4db-d64773638f70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:21.144224 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f672c1-4afd-4ba5-b4db-d64773638f70 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:21.178912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9699d3bf-3789-4d1e-8bf6-d5cb36837797 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:21.209928 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:21.217324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:21.220284 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1581c7-538f-4f7b-8fa6-8997a8237572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:21.221194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1581c7-538f-4f7b-8fa6-8997a8237572 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:21.221474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1581c7-538f-4f7b-8fa6-8997a8237572 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:21.221745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1581c7-538f-4f7b-8fa6-8997a8237572 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:21.269568 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 178/353] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:21.270455 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f672c1-4afd-4ba5-b4db-d64773638f70 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:21.300305 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1581c7-538f-4f7b-8fa6-8997a8237572 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:21.301785 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1581c7-538f-4f7b-8fa6-8997a8237572 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:21.395584 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f672c1-4afd-4ba5-b4db-d64773638f70 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:21.424044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 176/354] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:21.425316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e7f6a-b017-45bc-8a38-4a315c630b48 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:21.425607 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e7f6a-b017-45bc-8a38-4a315c630b48 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:21.425879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e7f6a-b017-45bc-8a38-4a315c630b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:21.440719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6808e8-4af6-489c-aaae-16ade28f2cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:21.441284 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6808e8-4af6-489c-aaae-16ade28f2cc5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:21.441559 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6808e8-4af6-489c-aaae-16ade28f2cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:21.441821 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6808e8-4af6-489c-aaae-16ade28f2cc5 None admin] PATH_INFO: `/v3/users/5f24fabc2ab74f8b87c7c9e0df49ae00` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:21.443017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6808e8-4af6-489c-aaae-16ade28f2cc5 None admin] RBAC: Authorizing `identity:get_user(user_id=5f24fabc2ab74f8b87c7c9e0df49ae00)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:21.445000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6808e8-4af6-489c-aaae-16ade28f2cc5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:21.446822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-363e7f6a-b017-45bc-8a38-4a315c630b48 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:21.446936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 179/355] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:13:21 2022] GET /identity/v3/users/5f24fabc2ab74f8b87c7c9e0df49ae00 => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:21.447744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714217a-1c45-4ed4-bdd6-5c3c6d958633 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:21.447844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-363e7f6a-b017-45bc-8a38-4a315c630b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:21.448025 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714217a-1c45-4ed4-bdd6-5c3c6d958633 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:21.448286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c714217a-1c45-4ed4-bdd6-5c3c6d958633 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:21.449820 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 180/356] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:21.450680 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714217a-1c45-4ed4-bdd6-5c3c6d958633 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:21.477291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:21.484644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:21.532732 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 177/357] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:21.532840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e7f6a-b017-45bc-8a38-4a315c630b48 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:21.576762 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c714217a-1c45-4ed4-bdd6-5c3c6d958633 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:21.622244 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a3f21-d29e-4829-8a26-48654dce7b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:21.622478 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a3f21-d29e-4829-8a26-48654dce7b23 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:21.622565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a3f21-d29e-4829-8a26-48654dce7b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:21.624704 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a3f21-d29e-4829-8a26-48654dce7b23 None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:21.624941 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a3f21-d29e-4829-8a26-48654dce7b23 None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:21.628438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a3f21-d29e-4829-8a26-48654dce7b23 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:21.628438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 181/358] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:21 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:21.630105 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a3f21-d29e-4829-8a26-48654dce7b23 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:21.669105 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e7f6a-b017-45bc-8a38-4a315c630b48 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:21.709783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457be804-4aeb-4e14-989f-0d6eacacd53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:21.710423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457be804-4aeb-4e14-989f-0d6eacacd53b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:21.710703 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457be804-4aeb-4e14-989f-0d6eacacd53b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:21.710997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457be804-4aeb-4e14-989f-0d6eacacd53b None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:21.712456 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457be804-4aeb-4e14-989f-0d6eacacd53b None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:21.713966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457be804-4aeb-4e14-989f-0d6eacacd53b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:21.715981 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 178/359] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:21 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:21.716839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457be804-4aeb-4e14-989f-0d6eacacd53b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:21.751394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:21.760097 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:21.768008 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a3f21-d29e-4829-8a26-48654dce7b23 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:21.814209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2aea6b-2c43-4204-a1ad-7c97dfd9b328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:21.814901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2aea6b-2c43-4204-a1ad-7c97dfd9b328 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:21.815185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2aea6b-2c43-4204-a1ad-7c97dfd9b328 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:21.815456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2aea6b-2c43-4204-a1ad-7c97dfd9b328 None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:21.816595 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2aea6b-2c43-4204-a1ad-7c97dfd9b328 None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:21.818051 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2aea6b-2c43-4204-a1ad-7c97dfd9b328 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:21.820773 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 182/360] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:21 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:21.821760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3247-6e49-4e8b-ad0a-01935bbdce1c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:21.822041 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3247-6e49-4e8b-ad0a-01935bbdce1c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:21.822341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f3247-6e49-4e8b-ad0a-01935bbdce1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:21.841236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3b9f3247-6e49-4e8b-ad0a-01935bbdce1c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:21.842490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3b9f3247-6e49-4e8b-ad0a-01935bbdce1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:21.854514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457be804-4aeb-4e14-989f-0d6eacacd53b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:21.900683 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc961b6-5e98-4e1a-b0eb-3275d5072c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:21.901547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc961b6-5e98-4e1a-b0eb-3275d5072c90 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:21.902055 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc961b6-5e98-4e1a-b0eb-3275d5072c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:21.902184 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc961b6-5e98-4e1a-b0eb-3275d5072c90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:21.902975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc961b6-5e98-4e1a-b0eb-3275d5072c90 None admin] RBAC: Adding query filter params (role.id=5a6ded16e6f54de78e356e7b984522fa, user.id=5f24fabc2ab74f8b87c7c9e0df49ae00, scope.system=all) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:21.903206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc961b6-5e98-4e1a-b0eb-3275d5072c90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:21.904665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc961b6-5e98-4e1a-b0eb-3275d5072c90 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:21.917733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 179/361] 10.222.0.76 () {56 vars in 1308 bytes} [Wed Aug 24 15:13:21 2022] GET /identity/v3/role_assignments?user.id=5f24fabc2ab74f8b87c7c9e0df49ae00&scope.system=all&role.id=5a6ded16e6f54de78e356e7b984522fa => generated 510 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:21.918606 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc961b6-5e98-4e1a-b0eb-3275d5072c90 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:21.921204 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 183/362] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:21.922070 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f3247-6e49-4e8b-ad0a-01935bbdce1c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:22.053790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:22.058036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:22.065362 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f3247-6e49-4e8b-ad0a-01935bbdce1c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:22.083889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc961b6-5e98-4e1a-b0eb-3275d5072c90 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:22.111996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc780fed-d16b-4758-bc7c-4a6dc5fbce27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:22.113105 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc780fed-d16b-4758-bc7c-4a6dc5fbce27 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:22.113563 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc780fed-d16b-4758-bc7c-4a6dc5fbce27 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:22.113834 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc780fed-d16b-4758-bc7c-4a6dc5fbce27 None admin] PATH_INFO: `/v3/groups/b7be5847e3ad4e739a0d54911f812f3d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:22.124427 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc780fed-d16b-4758-bc7c-4a6dc5fbce27 None admin] RBAC: Authorizing `identity:get_group(group_id=b7be5847e3ad4e739a0d54911f812f3d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:22.126490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc780fed-d16b-4758-bc7c-4a6dc5fbce27 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:22.128411 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 184/363] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 15:13:21 2022] GET /identity/v3/groups/b7be5847e3ad4e739a0d54911f812f3d => generated 226 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:22.129392 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc780fed-d16b-4758-bc7c-4a6dc5fbce27 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:22.135064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8566f92-d02e-4ee7-badd-daa080e0e7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:22.135794 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8566f92-d02e-4ee7-badd-daa080e0e7d3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:22.136158 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8566f92-d02e-4ee7-badd-daa080e0e7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:22.136433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8566f92-d02e-4ee7-badd-daa080e0e7d3 None admin] PATH_INFO: `/v3/groups/b7be5847e3ad4e739a0d54911f812f3d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:22.138200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8566f92-d02e-4ee7-badd-daa080e0e7d3 None admin] RBAC: Authorizing `identity:get_group(group_id=b7be5847e3ad4e739a0d54911f812f3d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:22.139679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8566f92-d02e-4ee7-badd-daa080e0e7d3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:22.141624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 180/364] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 15:13:21 2022] GET /identity/v3/groups/b7be5847e3ad4e739a0d54911f812f3d => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:22.142428 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8566f92-d02e-4ee7-badd-daa080e0e7d3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:22.258381 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc780fed-d16b-4758-bc7c-4a6dc5fbce27 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:22.262831 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8566f92-d02e-4ee7-badd-daa080e0e7d3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:22.305516 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea478c-f014-4532-b7d0-f9a79b04de34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:22.306094 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea478c-f014-4532-b7d0-f9a79b04de34 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:22.306504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea478c-f014-4532-b7d0-f9a79b04de34 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:22.306755 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea478c-f014-4532-b7d0-f9a79b04de34 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:22.317976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31808d5-2df3-4045-abb5-562d03ecfc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:22.319279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31808d5-2df3-4045-abb5-562d03ecfc9c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:22.320595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31808d5-2df3-4045-abb5-562d03ecfc9c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:22.320595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31808d5-2df3-4045-abb5-562d03ecfc9c None admin] PATH_INFO: `/v3/groups/b7be5847e3ad4e739a0d54911f812f3d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:22.321398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ea478c-f014-4532-b7d0-f9a79b04de34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:22.321580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31808d5-2df3-4045-abb5-562d03ecfc9c None admin] RBAC: Authorizing `identity:get_group(group_id=b7be5847e3ad4e739a0d54911f812f3d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:22.323035 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31808d5-2df3-4045-abb5-562d03ecfc9c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:22.324803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 181/365] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 15:13:22 2022] GET /identity/v3/groups/b7be5847e3ad4e739a0d54911f812f3d => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:22.325489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31808d5-2df3-4045-abb5-562d03ecfc9c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:22.327118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-54ea478c-f014-4532-b7d0-f9a79b04de34 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:22.327118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:22.327118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:22.327118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:22.327118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:22.327118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:22.327118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:22.327118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:22.327118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:22.327118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:22.327118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:22.327118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:22.327118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:22.327118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:22.327118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:22.327118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:22.328005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:22.328005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:22.328005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:22.328005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:22.328005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:22.328005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:22.328005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:22.328005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:22.328005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:22.328005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:22.328005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:22.328005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:22.328005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:22.328005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:22.328005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:22.328005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:22.328787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:22.328787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:22.328787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:22.328787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:22.328787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:22.328787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:22.328787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:22.328787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:22.328787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:22.328787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:22.328787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:22.328787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:22.329383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 185/366] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:13:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:22.330106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea478c-f014-4532-b7d0-f9a79b04de34 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:22.355483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:22.356289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:22.454406 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31808d5-2df3-4045-abb5-562d03ecfc9c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:22.458405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ea478c-f014-4532-b7d0-f9a79b04de34 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:22.498774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f22ef0-6e5c-4f47-b4c6-c7f2896cfbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:22.499231 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f22ef0-6e5c-4f47-b4c6-c7f2896cfbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:22.499508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f22ef0-6e5c-4f47-b4c6-c7f2896cfbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:22.499775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f22ef0-6e5c-4f47-b4c6-c7f2896cfbf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:22.500505 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f22ef0-6e5c-4f47-b4c6-c7f2896cfbf3 None admin] RBAC: Adding query filter params (group.id=b7be5847e3ad4e739a0d54911f812f3d, role.id=8d794ef8abb94809bd988c5db32c0037, scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:22.500810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f22ef0-6e5c-4f47-b4c6-c7f2896cfbf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:22.502673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f22ef0-6e5c-4f47-b4c6-c7f2896cfbf3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:22.502920 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e28541-e11e-4900-b97b-b911f3c836a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:22.503529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e28541-e11e-4900-b97b-b911f3c836a7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:22.503802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e28541-e11e-4900-b97b-b911f3c836a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:22.504392 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e28541-e11e-4900-b97b-b911f3c836a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:22.505422 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e28541-e11e-4900-b97b-b911f3c836a7 None admin] RBAC: Adding query filter params (group.id=b7be5847e3ad4e739a0d54911f812f3d, role.id=8d794ef8abb94809bd988c5db32c0037, scope.project.id=7e93b768826c4821ad3dd853154e62b4) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:22.505727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e28541-e11e-4900-b97b-b911f3c836a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:22.507888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e28541-e11e-4900-b97b-b911f3c836a7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:22.509445 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 182/367] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 15:13:22 2022] GET /identity/v3/role_assignments?group.id=b7be5847e3ad4e739a0d54911f812f3d&scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3&role.id=8d794ef8abb94809bd988c5db32c0037 => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:22.510861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f22ef0-6e5c-4f47-b4c6-c7f2896cfbf3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:22.515499 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 186/368] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 15:13:22 2022] GET /identity/v3/role_assignments?group.id=b7be5847e3ad4e739a0d54911f812f3d&scope.project.id=7e93b768826c4821ad3dd853154e62b4&role.id=8d794ef8abb94809bd988c5db32c0037 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:22.516190 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e28541-e11e-4900-b97b-b911f3c836a7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:22.635398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:22.647140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:22.659167 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e28541-e11e-4900-b97b-b911f3c836a7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:22.668717 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f22ef0-6e5c-4f47-b4c6-c7f2896cfbf3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:22.702053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82895a34-bde7-4b42-86c9-233afbc8ce0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:22.702815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82895a34-bde7-4b42-86c9-233afbc8ce0c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:22.703152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82895a34-bde7-4b42-86c9-233afbc8ce0c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:22.703500 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82895a34-bde7-4b42-86c9-233afbc8ce0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:22.704238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82895a34-bde7-4b42-86c9-233afbc8ce0c None admin] RBAC: Adding query filter params (group.id=b7be5847e3ad4e739a0d54911f812f3d, role.id=d123411b649d4acb8648d0c0b80aae6d, scope.project.id=7e93b768826c4821ad3dd853154e62b4) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:22.704540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82895a34-bde7-4b42-86c9-233afbc8ce0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:22.705985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82895a34-bde7-4b42-86c9-233afbc8ce0c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:22.713554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a87ee-3258-41fd-93b3-9b50c34a4225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:22.714036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 187/369] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 15:13:22 2022] GET /identity/v3/role_assignments?group.id=b7be5847e3ad4e739a0d54911f812f3d&scope.project.id=7e93b768826c4821ad3dd853154e62b4&role.id=d123411b649d4acb8648d0c0b80aae6d => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:22.714471 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a87ee-3258-41fd-93b3-9b50c34a4225 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:22.714754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a87ee-3258-41fd-93b3-9b50c34a4225 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:22.715024 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01a87ee-3258-41fd-93b3-9b50c34a4225 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:22.716345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82895a34-bde7-4b42-86c9-233afbc8ce0c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:22.730739 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01a87ee-3258-41fd-93b3-9b50c34a4225 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:22.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-e01a87ee-3258-41fd-93b3-9b50c34a4225 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:13:22.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:22.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:22.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:22.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:22.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:22.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:22.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:22.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:22.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:22.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:22.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:22.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:22.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:22.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:22.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:22.737245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:22.737245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:22.737245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:22.737245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:22.737245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:22.737245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:22.737245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:22.737245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:22.737245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:22.737245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:22.737245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:22.737245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:22.737245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:22.737245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:22.737245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:22.737245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:22.738016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:22.738016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:22.738016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:22.738016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:22.738016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:22.738016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:22.738016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:22.738016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:22.738016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:22.738016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:22.738016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:13:22.738016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:22.738016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 183/370] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:13:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:22.746103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a87ee-3258-41fd-93b3-9b50c34a4225 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:22.903739 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01a87ee-3258-41fd-93b3-9b50c34a4225 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:22.927760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82895a34-bde7-4b42-86c9-233afbc8ce0c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:22.974271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb79cb-d19b-4966-b725-eb0e3b9a8f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:22.975215 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb79cb-d19b-4966-b725-eb0e3b9a8f5e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:22.975495 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb79cb-d19b-4966-b725-eb0e3b9a8f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:22.976029 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb79cb-d19b-4966-b725-eb0e3b9a8f5e None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:22.977147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb79cb-d19b-4966-b725-eb0e3b9a8f5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:22.977644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb79cb-d19b-4966-b725-eb0e3b9a8f5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:22.980652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb79cb-d19b-4966-b725-eb0e3b9a8f5e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:22.986745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e609ab-7345-4c1a-957f-4b51769ef889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:22.987419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e609ab-7345-4c1a-957f-4b51769ef889 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:22.987699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e609ab-7345-4c1a-957f-4b51769ef889 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:22.987966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e609ab-7345-4c1a-957f-4b51769ef889 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:22.988633 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e609ab-7345-4c1a-957f-4b51769ef889 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:22.988934 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e609ab-7345-4c1a-957f-4b51769ef889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:22.990347 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e609ab-7345-4c1a-957f-4b51769ef889 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:22.992026 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 184/371] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:22.997615 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb79cb-d19b-4966-b725-eb0e3b9a8f5e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:23.004938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 188/372] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:23.013597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:23.020783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e609ab-7345-4c1a-957f-4b51769ef889 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:23.042770 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:23.164898 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb79cb-d19b-4966-b725-eb0e3b9a8f5e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:23.215157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e609ab-7345-4c1a-957f-4b51769ef889 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:23.234902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e37453-b7be-4e8b-8dc0-8d0f1d149eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:23.235523 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e37453-b7be-4e8b-8dc0-8d0f1d149eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:23.235795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e37453-b7be-4e8b-8dc0-8d0f1d149eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:23.236059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e37453-b7be-4e8b-8dc0-8d0f1d149eb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:23.252464 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898bec8-537f-4464-b3d4-a6386ff90b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:23.253181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898bec8-537f-4464-b3d4-a6386ff90b2d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:23.253463 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898bec8-537f-4464-b3d4-a6386ff90b2d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:23.253735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4898bec8-537f-4464-b3d4-a6386ff90b2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:23.335880 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898bec8-537f-4464-b3d4-a6386ff90b2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:23.337367 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4898bec8-537f-4464-b3d4-a6386ff90b2d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:23.348214 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:23.376663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e37453-b7be-4e8b-8dc0-8d0f1d149eb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:23.378722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e37453-b7be-4e8b-8dc0-8d0f1d149eb3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:23.395680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:23.468138 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 189/373] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 15:13:23.474369 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898bec8-537f-4464-b3d4-a6386ff90b2d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:23.514872 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 185/374] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:23.527432 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e37453-b7be-4e8b-8dc0-8d0f1d149eb3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:23.679661 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4898bec8-537f-4464-b3d4-a6386ff90b2d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:23.690869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:23.723414 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bb549-6c01-4b6e-9cd0-d6c30feabd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:23.723981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bb549-6c01-4b6e-9cd0-d6c30feabd91 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:23.724262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bb549-6c01-4b6e-9cd0-d6c30feabd91 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:23.724534 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37bb549-6c01-4b6e-9cd0-d6c30feabd91 None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:23.726695 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bb549-6c01-4b6e-9cd0-d6c30feabd91 None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:23.728267 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37bb549-6c01-4b6e-9cd0-d6c30feabd91 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:23.730045 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e37453-b7be-4e8b-8dc0-8d0f1d149eb3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:23.730424 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 190/375] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:23 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:23.736280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bb549-6c01-4b6e-9cd0-d6c30feabd91 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:23.746708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:23.791218 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee244f-e17a-4734-b570-369e6ce4cb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:23.792310 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee244f-e17a-4734-b570-369e6ce4cb63 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:23.792799 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee244f-e17a-4734-b570-369e6ce4cb63 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:23.793267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee244f-e17a-4734-b570-369e6ce4cb63 None admin] PATH_INFO: `/v3/projects/e70c51040e434074a3d61be0b7c8533b` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:23.795956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee244f-e17a-4734-b570-369e6ce4cb63 None admin] RBAC: Authorizing `identity:get_project(project_id=e70c51040e434074a3d61be0b7c8533b)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:23.797680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee244f-e17a-4734-b570-369e6ce4cb63 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:23.800489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 186/376] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:23 2022] GET /identity/v3/projects/e70c51040e434074a3d61be0b7c8533b => generated 344 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:23.806658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee244f-e17a-4734-b570-369e6ce4cb63 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:23.972061 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37bb549-6c01-4b6e-9cd0-d6c30feabd91 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:24.020775 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da512c10-971e-4427-8df0-2ef6de210cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:24.021327 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da512c10-971e-4427-8df0-2ef6de210cd3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:24.021638 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da512c10-971e-4427-8df0-2ef6de210cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:24.021903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da512c10-971e-4427-8df0-2ef6de210cd3 None admin] PATH_INFO: `/v3/groups/b7be5847e3ad4e739a0d54911f812f3d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:24.023053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da512c10-971e-4427-8df0-2ef6de210cd3 None admin] RBAC: Authorizing `identity:get_group(group_id=b7be5847e3ad4e739a0d54911f812f3d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:24.023485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee244f-e17a-4734-b570-369e6ce4cb63 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:24.024492 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da512c10-971e-4427-8df0-2ef6de210cd3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:24.027261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 191/377] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 15:13:23 2022] GET /identity/v3/groups/b7be5847e3ad4e739a0d54911f812f3d => generated 226 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:24.032625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da512c10-971e-4427-8df0-2ef6de210cd3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:24.061827 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228997b5-23d0-4057-897b-1a89b9c5167f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:24.062932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228997b5-23d0-4057-897b-1a89b9c5167f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:24.063420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228997b5-23d0-4057-897b-1a89b9c5167f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:24.063920 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228997b5-23d0-4057-897b-1a89b9c5167f None admin] PATH_INFO: `/v3/groups/80aaea5e9d6344c58f61a56d46607841` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:24.071689 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228997b5-23d0-4057-897b-1a89b9c5167f None admin] RBAC: Authorizing `identity:get_group(group_id=80aaea5e9d6344c58f61a56d46607841)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:24.073473 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228997b5-23d0-4057-897b-1a89b9c5167f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:24.076039 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 187/378] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 15:13:23 2022] GET /identity/v3/groups/80aaea5e9d6344c58f61a56d46607841 => generated 229 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:24.083420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228997b5-23d0-4057-897b-1a89b9c5167f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:24.083855 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:24.141672 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da512c10-971e-4427-8df0-2ef6de210cd3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:24.154898 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:24.182747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a42a550-9feb-44da-a225-d634377e52b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:24.183388 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42a550-9feb-44da-a225-d634377e52b5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:24.183641 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42a550-9feb-44da-a225-d634377e52b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:24.183923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a42a550-9feb-44da-a225-d634377e52b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:24.184640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42a550-9feb-44da-a225-d634377e52b5 None admin] RBAC: Adding query filter params (group.id=b7be5847e3ad4e739a0d54911f812f3d, role.id=d123411b649d4acb8648d0c0b80aae6d, scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:24.184945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42a550-9feb-44da-a225-d634377e52b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:24.186617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a42a550-9feb-44da-a225-d634377e52b5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:24.192812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 192/379] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 15:13:24 2022] GET /identity/v3/role_assignments?group.id=b7be5847e3ad4e739a0d54911f812f3d&scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3&role.id=d123411b649d4acb8648d0c0b80aae6d => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:24.250934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228997b5-23d0-4057-897b-1a89b9c5167f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:24.296648 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd783c3-d43f-4c11-b2ec-eb13bf66a9b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:24.304924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd783c3-d43f-4c11-b2ec-eb13bf66a9b7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:24.305444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd783c3-d43f-4c11-b2ec-eb13bf66a9b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:24.305974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd783c3-d43f-4c11-b2ec-eb13bf66a9b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:24.307001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd783c3-d43f-4c11-b2ec-eb13bf66a9b7 None admin] RBAC: Adding query filter params (group.id=80aaea5e9d6344c58f61a56d46607841, role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=e70c51040e434074a3d61be0b7c8533b) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:24.307537 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd783c3-d43f-4c11-b2ec-eb13bf66a9b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:24.309243 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd783c3-d43f-4c11-b2ec-eb13bf66a9b7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:24.318412 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 188/380] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 15:13:24 2022] GET /identity/v3/role_assignments?group.id=80aaea5e9d6344c58f61a56d46607841&scope.project.id=e70c51040e434074a3d61be0b7c8533b&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 260 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:24.883789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e053b-899b-4297-9ae9-3a318a722a1b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:24.884919 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e053b-899b-4297-9ae9-3a318a722a1b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:24.885405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e053b-899b-4297-9ae9-3a318a722a1b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:24.887257 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 193/381] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:24.892188 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fb53f-a3f9-44f4-9a37-415e2d9aa98a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:24.893034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fb53f-a3f9-44f4-9a37-415e2d9aa98a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:24.893516 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69fb53f-a3f9-44f4-9a37-415e2d9aa98a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:24.923182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c69fb53f-a3f9-44f4-9a37-415e2d9aa98a None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:24.924746 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c69fb53f-a3f9-44f4-9a37-415e2d9aa98a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:24.979485 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf1754-e397-4c4b-b9e1-d5da23055c98 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:24.979826 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf1754-e397-4c4b-b9e1-d5da23055c98 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:24.980037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf1754-e397-4c4b-b9e1-d5da23055c98 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:24.981708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 194/382] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:24.990239 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffaf00-8a9a-40a8-a28a-cbf033a810fc None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:24.990239 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffaf00-8a9a-40a8-a28a-cbf033a810fc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:24.990239 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ffaf00-8a9a-40a8-a28a-cbf033a810fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:25.013089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-18ffaf00-8a9a-40a8-a28a-cbf033a810fc None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:25.014337 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-18ffaf00-8a9a-40a8-a28a-cbf033a810fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:25.046413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 189/383] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:25.111955 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 195/384] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:13:25.294540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20565c16-27fd-4196-929a-cefc7f988cb4 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:25.295241 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20565c16-27fd-4196-929a-cefc7f988cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:25.295744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20565c16-27fd-4196-929a-cefc7f988cb4 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:25.297543 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 190/385] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:25.303379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62da04b-12b5-4ef5-bd1e-d1797aa88d0e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:25.303720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62da04b-12b5-4ef5-bd1e-d1797aa88d0e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:25.303969 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62da04b-12b5-4ef5-bd1e-d1797aa88d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:25.330614 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b62da04b-12b5-4ef5-bd1e-d1797aa88d0e None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:25.331892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b62da04b-12b5-4ef5-bd1e-d1797aa88d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:25.367938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:25.422722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 196/386] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:25.452139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20565c16-27fd-4196-929a-cefc7f988cb4 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:25.476441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62da04b-12b5-4ef5-bd1e-d1797aa88d0e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:25.495016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:25.590743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20565c16-27fd-4196-929a-cefc7f988cb4 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:25.629232 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d19ad-7e31-447d-a040-36623fe1ca13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:25.630272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d19ad-7e31-447d-a040-36623fe1ca13 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:25.630830 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d19ad-7e31-447d-a040-36623fe1ca13 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:25.631311 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d19ad-7e31-447d-a040-36623fe1ca13 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:25.643109 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d19ad-7e31-447d-a040-36623fe1ca13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:25.649053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-959d19ad-7e31-447d-a040-36623fe1ca13 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:25.649053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:25.649053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:25.649053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:25.649053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:25.649053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:25.649053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:25.649053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:25.649053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:25.649053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:25.649053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:25.649053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:25.649053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:25.649053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:25.649053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:25.649053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:25.649961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:25.649961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:25.649961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:25.649961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:25.649961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:25.649961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:25.649961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:25.649961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:25.649961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:25.649961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:25.649961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:25.649961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:25.649961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:25.649961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:25.649961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:25.649961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:25.651093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:25.651093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:25.651093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:25.651093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:25.651093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:25.651093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:25.651093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:25.651093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:25.651093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:25.651093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:25.651093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:25.651093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:25.654200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 191/387] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:25.660871 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d19ad-7e31-447d-a040-36623fe1ca13 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:25.741727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62da04b-12b5-4ef5-bd1e-d1797aa88d0e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:25.763324 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:25.774160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d19ad-7e31-447d-a040-36623fe1ca13 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:25.780895 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d60499-4c35-41c3-aaf5-9b50d31322f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:25.781738 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d60499-4c35-41c3-aaf5-9b50d31322f2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:25.782165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d60499-4c35-41c3-aaf5-9b50d31322f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:25.782608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d60499-4c35-41c3-aaf5-9b50d31322f2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:25.795022 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d60499-4c35-41c3-aaf5-9b50d31322f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:25.800297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-90d60499-4c35-41c3-aaf5-9b50d31322f2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:25.800297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:25.800297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:25.800297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:25.800297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:25.800297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:25.800297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:25.800297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:25.800297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:25.800297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:25.800297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:25.800297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:25.800297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:25.800297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:25.800297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:25.800297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:25.801173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:25.801173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:25.801173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:25.801173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:25.801173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:25.801173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:25.801173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:25.801173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:25.801173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:25.801173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:25.801173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:25.801173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:25.801173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:25.801173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:25.801173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:25.801173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:25.801943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:25.801943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:25.801943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:25.801943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:25.801943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:25.801943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:25.801943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:25.801943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:25.801943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:25.801943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:25.801943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:25.801943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:25.804566 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 197/388] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:25.805772 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d60499-4c35-41c3-aaf5-9b50d31322f2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:25.814373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362006f-c363-49c2-9cd5-8187b4369c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:25.815072 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362006f-c363-49c2-9cd5-8187b4369c94 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:25.815424 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362006f-c363-49c2-9cd5-8187b4369c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:25.815815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c362006f-c363-49c2-9cd5-8187b4369c94 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:25.816556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362006f-c363-49c2-9cd5-8187b4369c94 None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:25.816978 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362006f-c363-49c2-9cd5-8187b4369c94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:25.818657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c362006f-c363-49c2-9cd5-8187b4369c94 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:25.828376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 192/389] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:25.833920 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362006f-c363-49c2-9cd5-8187b4369c94 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:25.839668 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:25.919910 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d60499-4c35-41c3-aaf5-9b50d31322f2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:25.946665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c362006f-c363-49c2-9cd5-8187b4369c94 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:25.961550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b3c9a-1047-45a4-9e39-5acd3a961c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:25.962565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b3c9a-1047-45a4-9e39-5acd3a961c6a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:25.963120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b3c9a-1047-45a4-9e39-5acd3a961c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:25.963596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b3c9a-1047-45a4-9e39-5acd3a961c6a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:25.974878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075b3c9a-1047-45a4-9e39-5acd3a961c6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:25.980333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-075b3c9a-1047-45a4-9e39-5acd3a961c6a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:25.980333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:25.980333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:25.980333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:25.980333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:25.980333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:25.980333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:25.980333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:25.980333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:25.980333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:25.980333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:25.980333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:25.980333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:25.980333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:25.980333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:25.980333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:25.981177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:25.981177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:25.981177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:25.981177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:25.981177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:25.981177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:25.981177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:25.981177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:25.981177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:25.981177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:25.981177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:25.981177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:25.981177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:25.981177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:25.981177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:25.981177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:25.981944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:25.981944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:25.981944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:25.981944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:25.981944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:25.981944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:25.981944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:25.981944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:25.981944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:25.981944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:25.981944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:25.981944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:25.984191 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 198/390] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:13:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:25.985376 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b3c9a-1047-45a4-9e39-5acd3a961c6a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:25.986188 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b555dd9-5c52-43df-98ac-91b521d65fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:25.986940 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b555dd9-5c52-43df-98ac-91b521d65fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:25.987367 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b555dd9-5c52-43df-98ac-91b521d65fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:25.987736 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b555dd9-5c52-43df-98ac-91b521d65fa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:25.988551 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b555dd9-5c52-43df-98ac-91b521d65fa3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:25.989004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b555dd9-5c52-43df-98ac-91b521d65fa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:25.990535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b555dd9-5c52-43df-98ac-91b521d65fa3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:26.001063 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 193/391] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:26.002134 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b555dd9-5c52-43df-98ac-91b521d65fa3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:26.022823 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:26.086555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:26.101728 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b3c9a-1047-45a4-9e39-5acd3a961c6a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:26.109444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b555dd9-5c52-43df-98ac-91b521d65fa3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:26.139984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430890ae-025d-4ee5-83fb-817aac43fefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:26.140962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430890ae-025d-4ee5-83fb-817aac43fefb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:26.141373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430890ae-025d-4ee5-83fb-817aac43fefb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:26.141825 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430890ae-025d-4ee5-83fb-817aac43fefb None admin] PATH_INFO: `/v3/groups/b7be5847e3ad4e739a0d54911f812f3d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:26.143395 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430890ae-025d-4ee5-83fb-817aac43fefb None admin] RBAC: Authorizing `identity:get_group(group_id=b7be5847e3ad4e739a0d54911f812f3d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:26.145186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430890ae-025d-4ee5-83fb-817aac43fefb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:26.145834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110c88b-3f3c-483a-8524-30669778bcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:26.147251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110c88b-3f3c-483a-8524-30669778bcfe None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:26.147604 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110c88b-3f3c-483a-8524-30669778bcfe None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:26.147954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9110c88b-3f3c-483a-8524-30669778bcfe None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:26.148691 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110c88b-3f3c-483a-8524-30669778bcfe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:26.148906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 199/392] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 15:13:25 2022] GET /identity/v3/groups/b7be5847e3ad4e739a0d54911f812f3d => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:26.149893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430890ae-025d-4ee5-83fb-817aac43fefb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:26.153714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110c88b-3f3c-483a-8524-30669778bcfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:26.155294 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9110c88b-3f3c-483a-8524-30669778bcfe None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:26.164616 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 194/393] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:26.165341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110c88b-3f3c-483a-8524-30669778bcfe None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:26.252268 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:26.263755 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430890ae-025d-4ee5-83fb-817aac43fefb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:26.276943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9110c88b-3f3c-483a-8524-30669778bcfe None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:26.301767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63423428-cc49-45ab-9acb-fcaeb34826ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:26.302430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63423428-cc49-45ab-9acb-fcaeb34826ef None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:26.303247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63423428-cc49-45ab-9acb-fcaeb34826ef None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:26.303745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63423428-cc49-45ab-9acb-fcaeb34826ef None admin] PATH_INFO: `/v3/groups/b7be5847e3ad4e739a0d54911f812f3d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:26.305032 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63423428-cc49-45ab-9acb-fcaeb34826ef None admin] RBAC: Authorizing `identity:get_group(group_id=b7be5847e3ad4e739a0d54911f812f3d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:26.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63423428-cc49-45ab-9acb-fcaeb34826ef None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:26.308279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 200/394] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 15:13:26 2022] GET /identity/v3/groups/b7be5847e3ad4e739a0d54911f812f3d => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:26.313499 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63423428-cc49-45ab-9acb-fcaeb34826ef None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:26.316864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8bd5d-3d71-4f9e-a717-97b1fd6a0dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:26.317970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8bd5d-3d71-4f9e-a717-97b1fd6a0dfc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:26.319117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8bd5d-3d71-4f9e-a717-97b1fd6a0dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:26.319562 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8bd5d-3d71-4f9e-a717-97b1fd6a0dfc None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:26.321636 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8bd5d-3d71-4f9e-a717-97b1fd6a0dfc None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:26.324412 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8bd5d-3d71-4f9e-a717-97b1fd6a0dfc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:26.334200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 195/395] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:26 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:26.335414 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea2847-4bb2-4256-af5f-65ecebd9b2f0 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:26.335702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea2847-4bb2-4256-af5f-65ecebd9b2f0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:26.335983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdea2847-4bb2-4256-af5f-65ecebd9b2f0 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:26.337411 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 196/396] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:26.337953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea2847-4bb2-4256-af5f-65ecebd9b2f0 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:26.346708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:26.431375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63423428-cc49-45ab-9acb-fcaeb34826ef None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:26.454331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdea2847-4bb2-4256-af5f-65ecebd9b2f0 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:26.468348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073dcd71-aed7-42fe-8e16-124cf6441ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:26.468986 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073dcd71-aed7-42fe-8e16-124cf6441ea6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:26.469267 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073dcd71-aed7-42fe-8e16-124cf6441ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:26.469541 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073dcd71-aed7-42fe-8e16-124cf6441ea6 None admin] PATH_INFO: `/v3/groups/b7be5847e3ad4e739a0d54911f812f3d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:26.470688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073dcd71-aed7-42fe-8e16-124cf6441ea6 None admin] RBAC: Authorizing `identity:get_group(group_id=b7be5847e3ad4e739a0d54911f812f3d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:26.472185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073dcd71-aed7-42fe-8e16-124cf6441ea6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:26.473939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 201/397] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 15:13:26 2022] GET /identity/v3/groups/b7be5847e3ad4e739a0d54911f812f3d => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:26.478933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073dcd71-aed7-42fe-8e16-124cf6441ea6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:26.491478 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8347cae5-ef6e-4c02-8aa8-9bad2dcedb00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:26.491841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347cae5-ef6e-4c02-8aa8-9bad2dcedb00 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:26.492111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347cae5-ef6e-4c02-8aa8-9bad2dcedb00 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:26.492384 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8347cae5-ef6e-4c02-8aa8-9bad2dcedb00 None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:26.493512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347cae5-ef6e-4c02-8aa8-9bad2dcedb00 None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:26.494969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8347cae5-ef6e-4c02-8aa8-9bad2dcedb00 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:26.496598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 197/398] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:26 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:26.501017 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ef35c-11d3-492e-b048-4415cfc023f6 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:26.501292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ef35c-11d3-492e-b048-4415cfc023f6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:26.501567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ef35c-11d3-492e-b048-4415cfc023f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:26.519970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ec9ef35c-11d3-492e-b048-4415cfc023f6 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:26.521139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ec9ef35c-11d3-492e-b048-4415cfc023f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:26.537959 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:26.592428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:26.604482 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073dcd71-aed7-42fe-8e16-124cf6441ea6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:26.605061 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 198/399] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:26.605789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ef35c-11d3-492e-b048-4415cfc023f6 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:26.652698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b86881-85e1-41cd-8dab-e12471f0c09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:26.654400 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b86881-85e1-41cd-8dab-e12471f0c09f None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:26.654891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b86881-85e1-41cd-8dab-e12471f0c09f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:26.655357 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b86881-85e1-41cd-8dab-e12471f0c09f None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3/groups/b7be5847e3ad4e739a0d54911f812f3d/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:26.658406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b86881-85e1-41cd-8dab-e12471f0c09f None admin] RBAC: Authorizing `identity:create_grant(project_id=5a83c130b5ab41c69c2d74715f3c92d3, group_id=b7be5847e3ad4e739a0d54911f812f3d, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:26.660997 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b86881-85e1-41cd-8dab-e12471f0c09f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:26.679324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 202/400] 10.222.0.76 () {58 vars in 1442 bytes} [Wed Aug 24 15:13:26 2022] PUT /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3/groups/b7be5847e3ad4e739a0d54911f812f3d/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:13:26.680535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b86881-85e1-41cd-8dab-e12471f0c09f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:26.768370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ef35c-11d3-492e-b048-4415cfc023f6 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:26.810655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b86881-85e1-41cd-8dab-e12471f0c09f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:26.820891 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de777f63-7122-418a-93f9-f5becb16c3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:26.822439 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de777f63-7122-418a-93f9-f5becb16c3fb None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:26.823325 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de777f63-7122-418a-93f9-f5becb16c3fb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:26.823813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de777f63-7122-418a-93f9-f5becb16c3fb None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:26.825488 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de777f63-7122-418a-93f9-f5becb16c3fb None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:26.828150 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de777f63-7122-418a-93f9-f5becb16c3fb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:26.831654 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 199/401] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:26 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:26.833220 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e74af-6995-4b16-8556-7f9769ab14dd None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:26.833680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e74af-6995-4b16-8556-7f9769ab14dd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:26.834206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7e74af-6995-4b16-8556-7f9769ab14dd None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:26.836690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 200/402] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:26.838134 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e74af-6995-4b16-8556-7f9769ab14dd None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:26.854531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09269be-c8b3-4ba2-a55d-60fb8204dd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:26.855039 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09269be-c8b3-4ba2-a55d-60fb8204dd8b None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:26.855320 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09269be-c8b3-4ba2-a55d-60fb8204dd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:26.855594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09269be-c8b3-4ba2-a55d-60fb8204dd8b None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4/groups/b7be5847e3ad4e739a0d54911f812f3d/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:26.857742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09269be-c8b3-4ba2-a55d-60fb8204dd8b None admin] RBAC: Authorizing `identity:create_grant(project_id=7e93b768826c4821ad3dd853154e62b4, group_id=b7be5847e3ad4e739a0d54911f812f3d, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:26.859316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09269be-c8b3-4ba2-a55d-60fb8204dd8b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:26.860629 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:26.877155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 203/403] 10.222.0.76 () {58 vars in 1442 bytes} [Wed Aug 24 15:13:26 2022] PUT /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4/groups/b7be5847e3ad4e739a0d54911f812f3d/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:13:26.878576 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0480cfd-da8e-438b-a9e7-c5f5303e2372 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:26.878859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0480cfd-da8e-438b-a9e7-c5f5303e2372 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:26.879148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0480cfd-da8e-438b-a9e7-c5f5303e2372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:26.905764 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b0480cfd-da8e-438b-a9e7-c5f5303e2372 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:26.907022 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b0480cfd-da8e-438b-a9e7-c5f5303e2372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:26.916470 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:26.988222 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7e74af-6995-4b16-8556-7f9769ab14dd None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:27.006803 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 204/404] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:27.007774 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0480cfd-da8e-438b-a9e7-c5f5303e2372 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:27.045482 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847b4d3-fe9d-438a-a4a6-ea50ac48f265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:27.046675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847b4d3-fe9d-438a-a4a6-ea50ac48f265 None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:27.046954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847b4d3-fe9d-438a-a4a6-ea50ac48f265 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:27.047465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847b4d3-fe9d-438a-a4a6-ea50ac48f265 None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4/groups/b7be5847e3ad4e739a0d54911f812f3d/roles/d123411b649d4acb8648d0c0b80aae6d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:27.050454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847b4d3-fe9d-438a-a4a6-ea50ac48f265 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e93b768826c4821ad3dd853154e62b4, group_id=b7be5847e3ad4e739a0d54911f812f3d, role_id=d123411b649d4acb8648d0c0b80aae6d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:27.053347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847b4d3-fe9d-438a-a4a6-ea50ac48f265 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:27.071384 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 201/405] 10.222.0.76 () {58 vars in 1442 bytes} [Wed Aug 24 15:13:26 2022] PUT /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4/groups/b7be5847e3ad4e739a0d54911f812f3d/roles/d123411b649d4acb8648d0c0b80aae6d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:13:27.138664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0480cfd-da8e-438b-a9e7-c5f5303e2372 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:27.178762 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fddd3-77c3-4252-9b47-79b3ee5dc396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:27.179428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fddd3-77c3-4252-9b47-79b3ee5dc396 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:27.179702 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fddd3-77c3-4252-9b47-79b3ee5dc396 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:27.179973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fddd3-77c3-4252-9b47-79b3ee5dc396 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:27.191753 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fddd3-77c3-4252-9b47-79b3ee5dc396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:27.197518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7e1fddd3-77c3-4252-9b47-79b3ee5dc396 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:27.197518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:27.197518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:27.197518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:27.197518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:27.197518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:27.197518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:27.197518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:27.197518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:27.197518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:27.197518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:27.197518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:27.197518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:27.197518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:27.197518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:27.197518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:27.198483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:27.198483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:27.198483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:27.198483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:27.198483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:27.198483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:27.198483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:27.198483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:27.198483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:27.198483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:27.198483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:27.198483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:27.198483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:27.198483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:27.198483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:27.198483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:27.199254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:27.199254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:27.199254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:27.199254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:27.199254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:27.199254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:27.199254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:27.199254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:27.199254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:27.199254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:27.199254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:27.199254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:27.199254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 205/406] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:13:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:27.206896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847b4d3-fe9d-438a-a4a6-ea50ac48f265 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:27.313960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:27.336389 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fddd3-77c3-4252-9b47-79b3ee5dc396 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:27.340552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:27.355484 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847b4d3-fe9d-438a-a4a6-ea50ac48f265 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:27.421898 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b2558-c5ac-4f04-a4fa-e6e389e22f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:27.423131 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b2558-c5ac-4f04-a4fa-e6e389e22f25 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:27.423672 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b2558-c5ac-4f04-a4fa-e6e389e22f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:27.424240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b2558-c5ac-4f04-a4fa-e6e389e22f25 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:27.425281 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b2558-c5ac-4f04-a4fa-e6e389e22f25 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:27.426066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b2558-c5ac-4f04-a4fa-e6e389e22f25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:27.428969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b2558-c5ac-4f04-a4fa-e6e389e22f25 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:27.442147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 202/407] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:27.451626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b2558-c5ac-4f04-a4fa-e6e389e22f25 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:27.523573 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fddd3-77c3-4252-9b47-79b3ee5dc396 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:27.583166 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5dfa41-6534-4981-aed0-e8d7de0c8984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:27.583868 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dfa41-6534-4981-aed0-e8d7de0c8984 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:27.584743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dfa41-6534-4981-aed0-e8d7de0c8984 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:27.585015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5dfa41-6534-4981-aed0-e8d7de0c8984 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:27.609219 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5dfa41-6534-4981-aed0-e8d7de0c8984 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:27.620532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ef5dfa41-6534-4981-aed0-e8d7de0c8984 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:13:27.620532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:27.620532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:27.620532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:27.620532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:27.620532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:27.620532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:27.620532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:27.620532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:27.620532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:27.620532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:27.620532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:27.620532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:27.620532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:27.620532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:27.620532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:27.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:27.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:27.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:27.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:27.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:27.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:27.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:27.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:27.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:27.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:27.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:27.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:27.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:27.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:27.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:27.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:27.623021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:27.623021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:27.623021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:27.623021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:27.623021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:27.623021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:27.623021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:27.623021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:27.623021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:27.623021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:27.623021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:13:27.623021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:27.623021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 206/408] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:13:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:27.634171 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5dfa41-6534-4981-aed0-e8d7de0c8984 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:27.718133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b2558-c5ac-4f04-a4fa-e6e389e22f25 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:27.725643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:27.768824 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5dfa41-6534-4981-aed0-e8d7de0c8984 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:27.795212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba70a8-7298-43ca-9c87-5e2f87cc19f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:27.795847 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba70a8-7298-43ca-9c87-5e2f87cc19f9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:27.796124 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba70a8-7298-43ca-9c87-5e2f87cc19f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:27.796394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ba70a8-7298-43ca-9c87-5e2f87cc19f9 None admin] PATH_INFO: `/v3/groups/b7be5847e3ad4e739a0d54911f812f3d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:27.797535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba70a8-7298-43ca-9c87-5e2f87cc19f9 None admin] RBAC: Authorizing `identity:get_group(group_id=b7be5847e3ad4e739a0d54911f812f3d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:27.799014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ba70a8-7298-43ca-9c87-5e2f87cc19f9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:27.800584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 203/409] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 15:13:27 2022] GET /identity/v3/groups/b7be5847e3ad4e739a0d54911f812f3d => generated 226 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:27.822678 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba70a8-7298-43ca-9c87-5e2f87cc19f9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:27.833695 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316251c-5fa8-4c03-a5fa-69f3fb02c895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:27.834906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316251c-5fa8-4c03-a5fa-69f3fb02c895 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:27.835183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316251c-5fa8-4c03-a5fa-69f3fb02c895 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:27.835459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f316251c-5fa8-4c03-a5fa-69f3fb02c895 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:27.836005 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:27.836137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316251c-5fa8-4c03-a5fa-69f3fb02c895 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:27.836412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316251c-5fa8-4c03-a5fa-69f3fb02c895 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:27.838029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f316251c-5fa8-4c03-a5fa-69f3fb02c895 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:27.859725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 207/410] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:27.871246 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316251c-5fa8-4c03-a5fa-69f3fb02c895 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:27.950149 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ba70a8-7298-43ca-9c87-5e2f87cc19f9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:27.996502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f316251c-5fa8-4c03-a5fa-69f3fb02c895 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:28.003046 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebeab15-6bac-4750-b4b9-b0769de221b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:28.003726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeab15-6bac-4750-b4b9-b0769de221b7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:28.003979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeab15-6bac-4750-b4b9-b0769de221b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:28.004253 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebeab15-6bac-4750-b4b9-b0769de221b7 None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:28.005493 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebeab15-6bac-4750-b4b9-b0769de221b7 None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:28.007647 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebeab15-6bac-4750-b4b9-b0769de221b7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:28.009665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 204/411] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:27 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:28.024896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebeab15-6bac-4750-b4b9-b0769de221b7 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:28.035426 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:28.052942 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72cd62-ebcf-418a-9fbb-762cebd0335a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:28.053946 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72cd62-ebcf-418a-9fbb-762cebd0335a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:28.054490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72cd62-ebcf-418a-9fbb-762cebd0335a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:28.054957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a72cd62-ebcf-418a-9fbb-762cebd0335a None admin] PATH_INFO: `/v3/groups/80aaea5e9d6344c58f61a56d46607841` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:28.056328 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72cd62-ebcf-418a-9fbb-762cebd0335a None admin] RBAC: Authorizing `identity:get_group(group_id=80aaea5e9d6344c58f61a56d46607841)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:28.057967 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a72cd62-ebcf-418a-9fbb-762cebd0335a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:28.059927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 208/412] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 15:13:27 2022] GET /identity/v3/groups/80aaea5e9d6344c58f61a56d46607841 => generated 229 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:28.068737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72cd62-ebcf-418a-9fbb-762cebd0335a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:28.150295 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:28.188161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebeab15-6bac-4750-b4b9-b0769de221b7 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:28.232485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d7b25-adff-484d-86ca-345919cb12cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:28.233166 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d7b25-adff-484d-86ca-345919cb12cc None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:28.233441 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d7b25-adff-484d-86ca-345919cb12cc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:28.233709 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d7b25-adff-484d-86ca-345919cb12cc None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3/groups/b7be5847e3ad4e739a0d54911f812f3d/roles/d123411b649d4acb8648d0c0b80aae6d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:28.238559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d7b25-adff-484d-86ca-345919cb12cc None admin] RBAC: Authorizing `identity:create_grant(project_id=5a83c130b5ab41c69c2d74715f3c92d3, group_id=b7be5847e3ad4e739a0d54911f812f3d, role_id=d123411b649d4acb8648d0c0b80aae6d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:28.240265 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b01d7b25-adff-484d-86ca-345919cb12cc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:28.263959 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 205/413] 10.222.0.76 () {58 vars in 1442 bytes} [Wed Aug 24 15:13:28 2022] PUT /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3/groups/b7be5847e3ad4e739a0d54911f812f3d/roles/d123411b649d4acb8648d0c0b80aae6d => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:13:28.313066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a72cd62-ebcf-418a-9fbb-762cebd0335a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:28.370123 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24874221-f484-411a-a250-cbb93dc7fdb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:28.370824 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24874221-f484-411a-a250-cbb93dc7fdb1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:28.371105 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24874221-f484-411a-a250-cbb93dc7fdb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:28.371371 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24874221-f484-411a-a250-cbb93dc7fdb1 None admin] PATH_INFO: `/v3/projects/e70c51040e434074a3d61be0b7c8533b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:28.374923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24874221-f484-411a-a250-cbb93dc7fdb1 None admin] RBAC: Authorizing `identity:get_project(project_id=e70c51040e434074a3d61be0b7c8533b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:28.376360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24874221-f484-411a-a250-cbb93dc7fdb1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:28.378376 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 209/414] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:28 2022] GET /identity/v3/projects/e70c51040e434074a3d61be0b7c8533b => generated 344 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:28.384797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d7b25-adff-484d-86ca-345919cb12cc None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:28.535570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d7b25-adff-484d-86ca-345919cb12cc None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:28.571644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:28.580556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f749dfd0-a343-44a0-815d-fea82bc4460a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:28.581662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749dfd0-a343-44a0-815d-fea82bc4460a None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:28.581662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749dfd0-a343-44a0-815d-fea82bc4460a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:28.581829 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f749dfd0-a343-44a0-815d-fea82bc4460a None admin] PATH_INFO: `/v3/projects/e70c51040e434074a3d61be0b7c8533b/groups/80aaea5e9d6344c58f61a56d46607841/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:28.583835 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749dfd0-a343-44a0-815d-fea82bc4460a None admin] RBAC: Authorizing `identity:create_grant(project_id=e70c51040e434074a3d61be0b7c8533b, group_id=80aaea5e9d6344c58f61a56d46607841, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:28.585279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f749dfd0-a343-44a0-815d-fea82bc4460a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:28.601592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 206/415] 10.222.0.76 () {58 vars in 1442 bytes} [Wed Aug 24 15:13:28 2022] PUT /identity/v3/projects/e70c51040e434074a3d61be0b7c8533b/groups/80aaea5e9d6344c58f61a56d46607841/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:13:29.167329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a05814-5460-4fc1-8e87-a53efebb736c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:29.167881 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a05814-5460-4fc1-8e87-a53efebb736c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:29.167881 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a05814-5460-4fc1-8e87-a53efebb736c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:29.169414 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 210/416] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:29.174911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15ffdf-19ca-46be-8185-a07046ae3af0 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:29.175299 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15ffdf-19ca-46be-8185-a07046ae3af0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:29.175458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15ffdf-19ca-46be-8185-a07046ae3af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:29.199233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-bf15ffdf-19ca-46be-8185-a07046ae3af0 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:29.200484 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bf15ffdf-19ca-46be-8185-a07046ae3af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:29.264926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19440c3-f9f2-4db8-8647-9601d27c6f3f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:29.265233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19440c3-f9f2-4db8-8647-9601d27c6f3f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:29.265487 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19440c3-f9f2-4db8-8647-9601d27c6f3f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:29.267972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 211/417] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:29.275798 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6a0fc-802b-42a6-853d-55d66bb0bff9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:29.276086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6a0fc-802b-42a6-853d-55d66bb0bff9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:29.276360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6a0fc-802b-42a6-853d-55d66bb0bff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:29.278320 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 207/418] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:13:29.317380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b3c6a0fc-802b-42a6-853d-55d66bb0bff9 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:29.319201 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b3c6a0fc-802b-42a6-853d-55d66bb0bff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:29.401242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:29.418731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 212/419] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:29.602905 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15ffdf-19ca-46be-8185-a07046ae3af0 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:29.737739 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15ffdf-19ca-46be-8185-a07046ae3af0 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:29.767018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:29.784139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71ffee-7b5f-49be-a2f9-1475a48894cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:29.784780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71ffee-7b5f-49be-a2f9-1475a48894cc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:29.785059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71ffee-7b5f-49be-a2f9-1475a48894cc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:29.785332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b71ffee-7b5f-49be-a2f9-1475a48894cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:29.796413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b71ffee-7b5f-49be-a2f9-1475a48894cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:29.801314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-4b71ffee-7b5f-49be-a2f9-1475a48894cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:29.801314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:29.801314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:29.801314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:29.801314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:29.801314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:29.801314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:29.801314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:29.801314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:29.801314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:29.801314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:29.801314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:29.801314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:29.801314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:29.801314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:29.801314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:29.802177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:29.802177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:29.802177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:29.802177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:29.802177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:29.802177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:29.802177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:29.802177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:29.802177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:29.802177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:29.802177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:29.802177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:29.802177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:29.802177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:29.802177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:29.802177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:29.803775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:29.803775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:29.803775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:29.803775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:29.803775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:29.803775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:29.803775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:29.803775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:29.803775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:29.803775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:29.803775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:29.803775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:29.803775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 208/420] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:29.804904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6a0fc-802b-42a6-853d-55d66bb0bff9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:29.809720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71ffee-7b5f-49be-a2f9-1475a48894cc None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:29.933069 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b71ffee-7b5f-49be-a2f9-1475a48894cc None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:29.946457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6a0fc-802b-42a6-853d-55d66bb0bff9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:29.969410 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8617bd8d-ebc9-4bc8-bba0-8546c080be36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:29.970026 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617bd8d-ebc9-4bc8-bba0-8546c080be36 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:29.970344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617bd8d-ebc9-4bc8-bba0-8546c080be36 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:29.970621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8617bd8d-ebc9-4bc8-bba0-8546c080be36 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:29.971353 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617bd8d-ebc9-4bc8-bba0-8546c080be36 None admin] RBAC: Adding query filter params (name=member) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:29.971658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617bd8d-ebc9-4bc8-bba0-8546c080be36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:29.973042 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8617bd8d-ebc9-4bc8-bba0-8546c080be36 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:29.986404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 209/421] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:29.988154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b34cc3-c1e5-4ab9-a1e0-2bf841cd8308 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:29.988373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b34cc3-c1e5-4ab9-a1e0-2bf841cd8308 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:29.988641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b34cc3-c1e5-4ab9-a1e0-2bf841cd8308 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:29.989884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 210/422] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:29.991153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b34cc3-c1e5-4ab9-a1e0-2bf841cd8308 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:29.991830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275eca0-0fb7-4991-a7a3-b14dab551a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:29.992460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275eca0-0fb7-4991-a7a3-b14dab551a17 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:29.992729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275eca0-0fb7-4991-a7a3-b14dab551a17 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:29.992993 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275eca0-0fb7-4991-a7a3-b14dab551a17 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:30.008558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e275eca0-0fb7-4991-a7a3-b14dab551a17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:30.015445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e275eca0-0fb7-4991-a7a3-b14dab551a17 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:30.015445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:30.015445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:30.015445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:30.015445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:30.015445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:30.015445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:30.015445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:30.015445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:30.015445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:30.015445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:30.015445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:30.015445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:30.015445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:30.015445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:30.015445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:30.016892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:30.016892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:30.016892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:30.016892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:30.016892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:30.016892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:30.016892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:30.016892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:30.016892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:30.016892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:30.016892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:30.016892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:30.016892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:30.016892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:30.016892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:30.016892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:30.018673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:30.018673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:30.018673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:30.018673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:30.018673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:30.018673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:30.018673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:30.018673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:30.018673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:30.018673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:30.018673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 15:13:30.018673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:30.018673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 213/423] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:30.020292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a9ba6-dadc-43e7-8461-e3791344d9b3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:30.020292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a9ba6-dadc-43e7-8461-e3791344d9b3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:30.020292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6a9ba6-dadc-43e7-8461-e3791344d9b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:30.046658 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1f6a9ba6-dadc-43e7-8461-e3791344d9b3 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:30.048047 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1f6a9ba6-dadc-43e7-8461-e3791344d9b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:30.050173 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:30.111298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:30.120952 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b34cc3-c1e5-4ab9-a1e0-2bf841cd8308 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:30.142256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 214/424] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:30.142256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a9ba6-dadc-43e7-8461-e3791344d9b3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:30.207816 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc666650-2cfa-436f-8d19-6d02aab50849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:30.210032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc666650-2cfa-436f-8d19-6d02aab50849 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:30.210753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc666650-2cfa-436f-8d19-6d02aab50849 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:30.211280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc666650-2cfa-436f-8d19-6d02aab50849 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:30.257951 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6a9ba6-dadc-43e7-8461-e3791344d9b3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:30.306929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7d262-87fa-4ad3-bf6d-632392e881e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:30.307823 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7d262-87fa-4ad3-bf6d-632392e881e0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:30.308241 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7d262-87fa-4ad3-bf6d-632392e881e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:30.311682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7d262-87fa-4ad3-bf6d-632392e881e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:30.312524 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7d262-87fa-4ad3-bf6d-632392e881e0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:30.312973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7d262-87fa-4ad3-bf6d-632392e881e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:30.316821 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d7d262-87fa-4ad3-bf6d-632392e881e0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:30.331596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 215/425] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:13:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:30.337535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7d262-87fa-4ad3-bf6d-632392e881e0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:30.352108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc666650-2cfa-436f-8d19-6d02aab50849 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:30.353819 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc666650-2cfa-436f-8d19-6d02aab50849 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:30.405801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:30.442596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:30.524959 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 211/426] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:30.531777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc666650-2cfa-436f-8d19-6d02aab50849 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:30.563278 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d7d262-87fa-4ad3-bf6d-632392e881e0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:30.612389 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a80ed-4a3d-4511-a4b5-b6dd80d15e22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:30.613583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a80ed-4a3d-4511-a4b5-b6dd80d15e22 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:30.614743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a80ed-4a3d-4511-a4b5-b6dd80d15e22 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:30.615793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3a80ed-4a3d-4511-a4b5-b6dd80d15e22 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:30.654678 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc666650-2cfa-436f-8d19-6d02aab50849 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:30.695307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a80ed-4a3d-4511-a4b5-b6dd80d15e22 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:30.695511 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae4032-8e34-4e5a-ab83-3142d7cf6adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:30.695951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae4032-8e34-4e5a-ab83-3142d7cf6adc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:30.696227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae4032-8e34-4e5a-ab83-3142d7cf6adc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:30.696495 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fae4032-8e34-4e5a-ab83-3142d7cf6adc None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:30.697614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae4032-8e34-4e5a-ab83-3142d7cf6adc None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:30.698884 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3a80ed-4a3d-4511-a4b5-b6dd80d15e22 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:30.699058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fae4032-8e34-4e5a-ab83-3142d7cf6adc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:30.700840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 212/427] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:30 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:30.701577 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae4032-8e34-4e5a-ab83-3142d7cf6adc None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:30.715602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:30.746939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:30.809931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 216/428] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 15:13:30.810878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a80ed-4a3d-4511-a4b5-b6dd80d15e22 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:30.822868 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fae4032-8e34-4e5a-ab83-3142d7cf6adc None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:30.862617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d1185-97a2-4802-acd6-f2bd32286298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:30.863529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d1185-97a2-4802-acd6-f2bd32286298 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:30.863809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d1185-97a2-4802-acd6-f2bd32286298 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:30.864078 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d1185-97a2-4802-acd6-f2bd32286298 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:30.875088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d1185-97a2-4802-acd6-f2bd32286298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:30.879966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-6f5d1185-97a2-4802-acd6-f2bd32286298 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:30.879966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:30.879966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:30.879966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:30.879966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:30.879966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:30.879966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:30.879966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:30.879966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:30.879966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:30.879966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:30.879966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:30.879966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:30.879966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:30.879966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:30.879966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:30.881009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:30.881009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:30.881009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:30.881009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:30.881009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:30.881009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:30.881009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:30.881009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:30.881009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:30.881009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:30.881009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:30.881009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:30.881009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:30.881009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:30.881009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:30.881009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:30.882202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:30.882202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:30.882202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:30.882202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:30.882202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:30.882202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:30.882202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:30.882202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:30.882202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:30.882202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:30.882202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:30.882202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:30.882202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 213/429] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:13:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:30.882923 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb0225-711c-4258-9ae8-324c371e73d8 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:30.882996 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb0225-711c-4258-9ae8-324c371e73d8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:30.883319 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb0225-711c-4258-9ae8-324c371e73d8 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:30.886340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 214/430] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:30.886340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb0225-711c-4258-9ae8-324c371e73d8 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:30.931856 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3a80ed-4a3d-4511-a4b5-b6dd80d15e22 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:30.964489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:30.969562 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad22a93-d513-4e05-9dc3-a5ae64287344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:30.970273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad22a93-d513-4e05-9dc3-a5ae64287344 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:30.970604 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad22a93-d513-4e05-9dc3-a5ae64287344 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:30.970875 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad22a93-d513-4e05-9dc3-a5ae64287344 None admin] PATH_INFO: `/v3/groups/b7be5847e3ad4e739a0d54911f812f3d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:30.972036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad22a93-d513-4e05-9dc3-a5ae64287344 None admin] RBAC: Authorizing `identity:get_group(group_id=b7be5847e3ad4e739a0d54911f812f3d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:30.973502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad22a93-d513-4e05-9dc3-a5ae64287344 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:30.980500 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 217/431] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 15:13:30 2022] GET /identity/v3/groups/b7be5847e3ad4e739a0d54911f812f3d => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:30.980500 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad22a93-d513-4e05-9dc3-a5ae64287344 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:30.993645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:31.008205 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb0225-711c-4258-9ae8-324c371e73d8 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:31.052754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafda4df-df9c-49ee-ac81-01f12ac42870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:31.053446 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafda4df-df9c-49ee-ac81-01f12ac42870 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:31.053716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafda4df-df9c-49ee-ac81-01f12ac42870 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:31.053989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafda4df-df9c-49ee-ac81-01f12ac42870 None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:31.055150 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafda4df-df9c-49ee-ac81-01f12ac42870 None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:31.057413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafda4df-df9c-49ee-ac81-01f12ac42870 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:31.059075 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 215/432] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:30 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:31.060238 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd88921-3317-45b4-b7be-b69c392e6339 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:31.060458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd88921-3317-45b4-b7be-b69c392e6339 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:31.060725 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd88921-3317-45b4-b7be-b69c392e6339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:31.085039 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cdd88921-3317-45b4-b7be-b69c392e6339 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:31.086311 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cdd88921-3317-45b4-b7be-b69c392e6339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:31.108243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad22a93-d513-4e05-9dc3-a5ae64287344 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:31.144114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721cbf1-17ba-4e11-a6fe-455b694dd3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:31.144718 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721cbf1-17ba-4e11-a6fe-455b694dd3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:31.144995 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721cbf1-17ba-4e11-a6fe-455b694dd3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:31.145265 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3721cbf1-17ba-4e11-a6fe-455b694dd3a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:31.145932 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721cbf1-17ba-4e11-a6fe-455b694dd3a4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:31.146281 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721cbf1-17ba-4e11-a6fe-455b694dd3a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:31.148175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3721cbf1-17ba-4e11-a6fe-455b694dd3a4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:31.156484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 218/433] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:31.157208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721cbf1-17ba-4e11-a6fe-455b694dd3a4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:31.163776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 216/434] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:31.165678 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba560a6-fcf7-4e24-982c-fc67d00001f2 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:31.165971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba560a6-fcf7-4e24-982c-fc67d00001f2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:31.166257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba560a6-fcf7-4e24-982c-fc67d00001f2 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:31.168589 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 217/435] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:31.168589 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba560a6-fcf7-4e24-982c-fc67d00001f2 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:31.238775 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:31.263258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:31.277077 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3721cbf1-17ba-4e11-a6fe-455b694dd3a4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:31.306150 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba560a6-fcf7-4e24-982c-fc67d00001f2 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:31.314745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39617e-adb3-4735-82ed-e0f1a463b3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:31.315288 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39617e-adb3-4735-82ed-e0f1a463b3ca None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:31.315567 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39617e-adb3-4735-82ed-e0f1a463b3ca None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:31.315844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d39617e-adb3-4735-82ed-e0f1a463b3ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:31.316677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d39617e-adb3-4735-82ed-e0f1a463b3ca None admin] RBAC: Adding query filter params (group.id=b7be5847e3ad4e739a0d54911f812f3d, role.id=8d794ef8abb94809bd988c5db32c0037, scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:31.316995 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d39617e-adb3-4735-82ed-e0f1a463b3ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:31.319021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d39617e-adb3-4735-82ed-e0f1a463b3ca None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:31.325203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 219/436] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 15:13:31 2022] GET /identity/v3/role_assignments?group.id=b7be5847e3ad4e739a0d54911f812f3d&scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3&role.id=8d794ef8abb94809bd988c5db32c0037 => generated 611 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:31.326008 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39617e-adb3-4735-82ed-e0f1a463b3ca None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:31.365397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce6417b-d3dc-4042-9663-5831d47ffc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:31.365990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6417b-d3dc-4042-9663-5831d47ffc68 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:31.366361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6417b-d3dc-4042-9663-5831d47ffc68 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:31.366635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce6417b-d3dc-4042-9663-5831d47ffc68 None admin] PATH_INFO: `/v3/groups/b7be5847e3ad4e739a0d54911f812f3d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:31.367915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6417b-d3dc-4042-9663-5831d47ffc68 None admin] RBAC: Authorizing `identity:get_group(group_id=b7be5847e3ad4e739a0d54911f812f3d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:31.369433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce6417b-d3dc-4042-9663-5831d47ffc68 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:31.377108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 218/437] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 15:13:31 2022] GET /identity/v3/groups/b7be5847e3ad4e739a0d54911f812f3d => generated 226 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:31.378596 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64161c-5503-4032-9960-90a839089479 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:31.378977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64161c-5503-4032-9960-90a839089479 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:31.379327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64161c-5503-4032-9960-90a839089479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:31.404266 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3a64161c-5503-4032-9960-90a839089479 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:31.406126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3a64161c-5503-4032-9960-90a839089479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:31.458146 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d39617e-adb3-4735-82ed-e0f1a463b3ca None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:31.496993 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 219/438] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:31.497696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64161c-5503-4032-9960-90a839089479 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:31.504432 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50a3bb-b704-4800-a6d8-d12f405d984b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:31.505051 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50a3bb-b704-4800-a6d8-d12f405d984b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:31.505330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50a3bb-b704-4800-a6d8-d12f405d984b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:31.505627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50a3bb-b704-4800-a6d8-d12f405d984b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:31.510887 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:31.595830 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:31.618471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de50a3bb-b704-4800-a6d8-d12f405d984b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:31.620071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de50a3bb-b704-4800-a6d8-d12f405d984b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:31.647768 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64161c-5503-4032-9960-90a839089479 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:31.690261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1089a9-b644-40ff-b103-2ad795c3d7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:31.690261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1089a9-b644-40ff-b103-2ad795c3d7fe None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:31.690261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1089a9-b644-40ff-b103-2ad795c3d7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:31.690616 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1089a9-b644-40ff-b103-2ad795c3d7fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:31.691254 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1089a9-b644-40ff-b103-2ad795c3d7fe None admin] RBAC: Adding query filter params (group.id=b7be5847e3ad4e739a0d54911f812f3d, role.id=8d794ef8abb94809bd988c5db32c0037, scope.project.id=7e93b768826c4821ad3dd853154e62b4) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:31.691563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1089a9-b644-40ff-b103-2ad795c3d7fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:31.693009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1089a9-b644-40ff-b103-2ad795c3d7fe None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:31.710252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 220/439] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 15:13:31 2022] GET /identity/v3/role_assignments?group.id=b7be5847e3ad4e739a0d54911f812f3d&scope.project.id=7e93b768826c4821ad3dd853154e62b4&role.id=8d794ef8abb94809bd988c5db32c0037 => generated 611 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:31.710252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1089a9-b644-40ff-b103-2ad795c3d7fe None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:31.761503 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 220/440] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:31.767327 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50a3bb-b704-4800-a6d8-d12f405d984b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:31.810603 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:31.894729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1089a9-b644-40ff-b103-2ad795c3d7fe None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:31.895115 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50a3bb-b704-4800-a6d8-d12f405d984b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:31.934805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:31.941904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f1cbd-356e-40b5-8fb8-069e16752b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:31.942570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f1cbd-356e-40b5-8fb8-069e16752b3e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:31.942889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f1cbd-356e-40b5-8fb8-069e16752b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:31.943907 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8f1cbd-356e-40b5-8fb8-069e16752b3e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:31.946977 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c08f4f-440d-435f-82d0-9737ec9395cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:31.948026 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c08f4f-440d-435f-82d0-9737ec9395cb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:31.948505 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c08f4f-440d-435f-82d0-9737ec9395cb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:31.948970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c08f4f-440d-435f-82d0-9737ec9395cb None admin] PATH_INFO: `/v3/projects/7e93b768826c4821ad3dd853154e62b4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:31.952564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c08f4f-440d-435f-82d0-9737ec9395cb None admin] RBAC: Authorizing `identity:get_project(project_id=7e93b768826c4821ad3dd853154e62b4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:31.956299 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8f1cbd-356e-40b5-8fb8-069e16752b3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:31.957205 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c08f4f-440d-435f-82d0-9737ec9395cb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:31.960741 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 221/441] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:31 2022] GET /identity/v3/projects/7e93b768826c4821ad3dd853154e62b4 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:31.962099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-7a8f1cbd-356e-40b5-8fb8-069e16752b3e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:31.962099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:31.962099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:31.962099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:31.962099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:31.962099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:31.962099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:31.962099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:31.962099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:31.962099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:31.962099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:31.962099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:31.962099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:31.962099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:31.962099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:31.962099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:31.963082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:31.963082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:31.963082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:31.963082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:31.963082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:31.963082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:31.963082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:31.963082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:31.963082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:31.963082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:31.963082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:31.963082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:31.963082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:31.963082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:31.963082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:31.963082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:31.963943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:31.963943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:31.963943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:31.963943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:31.963943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:31.963943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:31.963943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:31.963943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:31.963943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:31.963943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:31.963943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:31.963943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:31.964590 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c08f4f-440d-435f-82d0-9737ec9395cb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:31.964658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 221/442] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:13:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:31.969646 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f1cbd-356e-40b5-8fb8-069e16752b3e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:32.096253 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8f1cbd-356e-40b5-8fb8-069e16752b3e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:32.096597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c08f4f-440d-435f-82d0-9737ec9395cb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:32.100252 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:32.135052 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1403dbe-a63e-49d7-9f8f-bcd3f46b4b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:32.135294 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286d8d7-5ef2-4c63-90a9-07a45d5a86e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:32.135681 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1403dbe-a63e-49d7-9f8f-bcd3f46b4b1d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:32.135958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1403dbe-a63e-49d7-9f8f-bcd3f46b4b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:32.136230 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286d8d7-5ef2-4c63-90a9-07a45d5a86e0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:32.136327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1403dbe-a63e-49d7-9f8f-bcd3f46b4b1d None admin] PATH_INFO: `/v3/groups/b7be5847e3ad4e739a0d54911f812f3d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:32.136534 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286d8d7-5ef2-4c63-90a9-07a45d5a86e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:32.136805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9286d8d7-5ef2-4c63-90a9-07a45d5a86e0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:32.137532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1403dbe-a63e-49d7-9f8f-bcd3f46b4b1d None admin] RBAC: Authorizing `identity:get_group(group_id=b7be5847e3ad4e739a0d54911f812f3d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:32.140183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1403dbe-a63e-49d7-9f8f-bcd3f46b4b1d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:32.141772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 222/443] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 15:13:31 2022] GET /identity/v3/groups/b7be5847e3ad4e739a0d54911f812f3d => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:32.148942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1403dbe-a63e-49d7-9f8f-bcd3f46b4b1d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:32.149077 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9286d8d7-5ef2-4c63-90a9-07a45d5a86e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:32.154655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-9286d8d7-5ef2-4c63-90a9-07a45d5a86e0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:13:32.154655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:32.154655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:32.154655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:32.154655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:32.154655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:32.154655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:32.154655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:32.154655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:32.154655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:32.154655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:32.154655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:32.154655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:32.154655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:32.154655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:32.154655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:32.155751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:32.155751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:32.155751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:32.155751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:32.155751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:32.155751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:32.155751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:32.155751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:32.155751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:32.155751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:32.155751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:32.155751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:32.155751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:32.155751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:32.155751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:32.155751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:32.156574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:32.156574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:32.156574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:32.156574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:32.156574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:32.156574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:32.156574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:32.156574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:32.156574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:32.156574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:32.156574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:13:32.156574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:32.156574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 222/444] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:13:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:32.157253 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286d8d7-5ef2-4c63-90a9-07a45d5a86e0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:32.216581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:32.266044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9286d8d7-5ef2-4c63-90a9-07a45d5a86e0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:32.283779 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1403dbe-a63e-49d7-9f8f-bcd3f46b4b1d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:32.301563 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03543600-a3bf-4bf9-86ad-4a19f493db75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:32.302171 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03543600-a3bf-4bf9-86ad-4a19f493db75 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:32.303040 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03543600-a3bf-4bf9-86ad-4a19f493db75 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:32.303304 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03543600-a3bf-4bf9-86ad-4a19f493db75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:32.304028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03543600-a3bf-4bf9-86ad-4a19f493db75 None admin] RBAC: Adding query filter params (group.id=b7be5847e3ad4e739a0d54911f812f3d, role.id=d123411b649d4acb8648d0c0b80aae6d, scope.project.id=7e93b768826c4821ad3dd853154e62b4) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:32.304333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03543600-a3bf-4bf9-86ad-4a19f493db75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:32.305781 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03543600-a3bf-4bf9-86ad-4a19f493db75 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:32.313039 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 223/445] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 15:13:32 2022] GET /identity/v3/role_assignments?group.id=b7be5847e3ad4e739a0d54911f812f3d&scope.project.id=7e93b768826c4821ad3dd853154e62b4&role.id=d123411b649d4acb8648d0c0b80aae6d => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:32.313859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03543600-a3bf-4bf9-86ad-4a19f493db75 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:32.323332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2912e3-8bb1-4761-9d6a-306f597fa7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:32.324233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2912e3-8bb1-4761-9d6a-306f597fa7de None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:32.324513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2912e3-8bb1-4761-9d6a-306f597fa7de None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:32.324782 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2912e3-8bb1-4761-9d6a-306f597fa7de None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:32.325481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2912e3-8bb1-4761-9d6a-306f597fa7de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:32.325793 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2912e3-8bb1-4761-9d6a-306f597fa7de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:32.327270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2912e3-8bb1-4761-9d6a-306f597fa7de None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:32.337976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 223/446] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:32.344612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2912e3-8bb1-4761-9d6a-306f597fa7de None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:32.358266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:32.444815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03543600-a3bf-4bf9-86ad-4a19f493db75 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:32.478059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2912e3-8bb1-4761-9d6a-306f597fa7de None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:32.482770 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15402ac0-125f-4df2-958e-9356fc2087cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:32.483434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15402ac0-125f-4df2-958e-9356fc2087cf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:32.483712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15402ac0-125f-4df2-958e-9356fc2087cf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:32.483993 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15402ac0-125f-4df2-958e-9356fc2087cf None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:32.484636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15402ac0-125f-4df2-958e-9356fc2087cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:32.484968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15402ac0-125f-4df2-958e-9356fc2087cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:32.486991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15402ac0-125f-4df2-958e-9356fc2087cf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:32.497946 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 224/447] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:32.504968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15402ac0-125f-4df2-958e-9356fc2087cf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:32.508461 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:32.543121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2780a6-07d2-4979-b6ae-7ff7d779eed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:32.544256 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2780a6-07d2-4979-b6ae-7ff7d779eed4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:32.544711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2780a6-07d2-4979-b6ae-7ff7d779eed4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:32.545191 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2780a6-07d2-4979-b6ae-7ff7d779eed4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:32.640311 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2780a6-07d2-4979-b6ae-7ff7d779eed4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:32.642378 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2780a6-07d2-4979-b6ae-7ff7d779eed4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:32.648721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:32.664919 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15402ac0-125f-4df2-958e-9356fc2087cf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:32.711903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed1fbf-c3c1-4c90-b6fa-9df2078c6e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:32.712623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed1fbf-c3c1-4c90-b6fa-9df2078c6e99 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:32.712991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed1fbf-c3c1-4c90-b6fa-9df2078c6e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:32.713182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fed1fbf-c3c1-4c90-b6fa-9df2078c6e99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:32.773897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 224/448] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 15:13:32.779958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2780a6-07d2-4979-b6ae-7ff7d779eed4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:32.805221 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed1fbf-c3c1-4c90-b6fa-9df2078c6e99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:32.806729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fed1fbf-c3c1-4c90-b6fa-9df2078c6e99 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:32.808112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:32.908525 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2780a6-07d2-4979-b6ae-7ff7d779eed4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:32.917380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:32.929158 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 225/449] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:32.935104 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed1fbf-c3c1-4c90-b6fa-9df2078c6e99 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:32.950845 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01721ad5-efbe-4b46-bd02-e58abe0fc946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:32.951458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01721ad5-efbe-4b46-bd02-e58abe0fc946 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:32.951731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01721ad5-efbe-4b46-bd02-e58abe0fc946 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:32.951996 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01721ad5-efbe-4b46-bd02-e58abe0fc946 None admin] PATH_INFO: `/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:32.953261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01721ad5-efbe-4b46-bd02-e58abe0fc946 None admin] RBAC: Authorizing `identity:get_project(project_id=5a83c130b5ab41c69c2d74715f3c92d3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:32.954722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01721ad5-efbe-4b46-bd02-e58abe0fc946 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:32.956328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 225/450] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:32 2022] GET /identity/v3/projects/5a83c130b5ab41c69c2d74715f3c92d3 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:32.962805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01721ad5-efbe-4b46-bd02-e58abe0fc946 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:33.036107 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fed1fbf-c3c1-4c90-b6fa-9df2078c6e99 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:33.056415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:33.068902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01721ad5-efbe-4b46-bd02-e58abe0fc946 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:33.073199 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbecfab-e5ee-4c05-ae76-21f2c457314b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:33.073831 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbecfab-e5ee-4c05-ae76-21f2c457314b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:33.074106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbecfab-e5ee-4c05-ae76-21f2c457314b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:33.074405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbecfab-e5ee-4c05-ae76-21f2c457314b None admin] PATH_INFO: `/v3/projects/e70c51040e434074a3d61be0b7c8533b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:33.075570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbecfab-e5ee-4c05-ae76-21f2c457314b None admin] RBAC: Authorizing `identity:get_project(project_id=e70c51040e434074a3d61be0b7c8533b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:33.076984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbecfab-e5ee-4c05-ae76-21f2c457314b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:33.079124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 226/451] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:13:32 2022] GET /identity/v3/projects/e70c51040e434074a3d61be0b7c8533b => generated 344 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:33.084764 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbecfab-e5ee-4c05-ae76-21f2c457314b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:33.109353 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a96e1-4c6b-48e7-bb1f-7dff199012d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:33.109978 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a96e1-4c6b-48e7-bb1f-7dff199012d6 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:33.110282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a96e1-4c6b-48e7-bb1f-7dff199012d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:33.110553 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a96e1-4c6b-48e7-bb1f-7dff199012d6 None admin] PATH_INFO: `/v3/groups/b7be5847e3ad4e739a0d54911f812f3d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:33.112088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a96e1-4c6b-48e7-bb1f-7dff199012d6 None admin] RBAC: Authorizing `identity:get_group(group_id=b7be5847e3ad4e739a0d54911f812f3d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:33.113536 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a96e1-4c6b-48e7-bb1f-7dff199012d6 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:33.115667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 226/452] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 15:13:32 2022] GET /identity/v3/groups/b7be5847e3ad4e739a0d54911f812f3d => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:33.121759 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a96e1-4c6b-48e7-bb1f-7dff199012d6 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:33.152213 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:33.200705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbecfab-e5ee-4c05-ae76-21f2c457314b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:33.233053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a96e1-4c6b-48e7-bb1f-7dff199012d6 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:33.239193 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11dd81-1764-4302-bad4-3e510ff7bf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:33.239692 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11dd81-1764-4302-bad4-3e510ff7bf11 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:33.240460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11dd81-1764-4302-bad4-3e510ff7bf11 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:33.240734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11dd81-1764-4302-bad4-3e510ff7bf11 None admin] PATH_INFO: `/v3/groups/80aaea5e9d6344c58f61a56d46607841` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:33.241866 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11dd81-1764-4302-bad4-3e510ff7bf11 None admin] RBAC: Authorizing `identity:get_group(group_id=80aaea5e9d6344c58f61a56d46607841)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:33.243389 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b11dd81-1764-4302-bad4-3e510ff7bf11 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:33.244970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 227/453] 10.222.0.76 () {56 vars in 1180 bytes} [Wed Aug 24 15:13:33 2022] GET /identity/v3/groups/80aaea5e9d6344c58f61a56d46607841 => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:33.250991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11dd81-1764-4302-bad4-3e510ff7bf11 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:33.269670 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890898ba-03dd-4098-9416-5f505cf6c384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:33.270308 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890898ba-03dd-4098-9416-5f505cf6c384 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:33.270584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890898ba-03dd-4098-9416-5f505cf6c384 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:33.270849 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890898ba-03dd-4098-9416-5f505cf6c384 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:33.271606 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890898ba-03dd-4098-9416-5f505cf6c384 None admin] RBAC: Adding query filter params (group.id=b7be5847e3ad4e739a0d54911f812f3d, role.id=d123411b649d4acb8648d0c0b80aae6d, scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:33.271910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890898ba-03dd-4098-9416-5f505cf6c384 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:33.273355 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890898ba-03dd-4098-9416-5f505cf6c384 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:33.279842 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 227/454] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 15:13:33 2022] GET /identity/v3/role_assignments?group.id=b7be5847e3ad4e739a0d54911f812f3d&scope.project.id=5a83c130b5ab41c69c2d74715f3c92d3&role.id=d123411b649d4acb8648d0c0b80aae6d => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:33.366918 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11dd81-1764-4302-bad4-3e510ff7bf11 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:33.396696 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:33.409346 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50368d56-42cc-413a-9051-3a6a112b222f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:33.410146 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50368d56-42cc-413a-9051-3a6a112b222f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:33.410492 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50368d56-42cc-413a-9051-3a6a112b222f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:33.410767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50368d56-42cc-413a-9051-3a6a112b222f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:33.412371 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50368d56-42cc-413a-9051-3a6a112b222f None admin] RBAC: Adding query filter params (group.id=80aaea5e9d6344c58f61a56d46607841, role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=e70c51040e434074a3d61be0b7c8533b) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:33.413092 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50368d56-42cc-413a-9051-3a6a112b222f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:33.414150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50368d56-42cc-413a-9051-3a6a112b222f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:33.421197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 228/455] 10.222.0.76 () {56 vars in 1376 bytes} [Wed Aug 24 15:13:33 2022] GET /identity/v3/role_assignments?group.id=80aaea5e9d6344c58f61a56d46607841&scope.project.id=e70c51040e434074a3d61be0b7c8533b&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:35.856141 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2890b9-5574-4ffa-8612-92f4ddb7dda4 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:35.856639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2890b9-5574-4ffa-8612-92f4ddb7dda4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:35.856639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2890b9-5574-4ffa-8612-92f4ddb7dda4 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:35.858188 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 228/456] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:35.870145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8f987-d59d-4320-9248-d86a7533c1bb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:35.870694 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8f987-d59d-4320-9248-d86a7533c1bb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:35.870770 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a8f987-d59d-4320-9248-d86a7533c1bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:35.895233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-16a8f987-d59d-4320-9248-d86a7533c1bb None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:35.896538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-16a8f987-d59d-4320-9248-d86a7533c1bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:35.987921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 229/457] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:36.012522 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9df21b-493a-4a6b-82ee-08263d3654ee None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:36.013215 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9df21b-493a-4a6b-82ee-08263d3654ee None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:36.013686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9df21b-493a-4a6b-82ee-08263d3654ee None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:36.015684 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 229/458] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:36.021684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126176d6-dfbb-429f-9b98-97cf99d59d38 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:36.021972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126176d6-dfbb-429f-9b98-97cf99d59d38 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:36.022288 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126176d6-dfbb-429f-9b98-97cf99d59d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:36.051237 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-126176d6-dfbb-429f-9b98-97cf99d59d38 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:36.052518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-126176d6-dfbb-429f-9b98-97cf99d59d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:36.134163 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 230/459] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:36.373263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9df21b-493a-4a6b-82ee-08263d3654ee None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:36.406576 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:36.456671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126176d6-dfbb-429f-9b98-97cf99d59d38 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:36.539714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9df21b-493a-4a6b-82ee-08263d3654ee None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:36.551655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:36.579335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab2d27-81fe-4bca-98a2-06c71a59ef92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:36.580053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab2d27-81fe-4bca-98a2-06c71a59ef92 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:36.580336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab2d27-81fe-4bca-98a2-06c71a59ef92 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:36.580657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab2d27-81fe-4bca-98a2-06c71a59ef92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:36.592205 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab2d27-81fe-4bca-98a2-06c71a59ef92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:36.593663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ab2d27-81fe-4bca-98a2-06c71a59ef92 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:36.611674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-10ab2d27-81fe-4bca-98a2-06c71a59ef92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:36.611674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:36.611674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:36.611674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:36.611674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:36.611674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:36.611674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:36.611674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:36.611674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:36.611674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:36.611674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:36.611674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:36.611674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:36.611674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:36.611674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:36.611674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:36.612655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:36.613924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:36.613924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:36.613924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:36.613924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 230/460] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:36.615916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08360e-ee67-4621-8f1e-cc1e173c3815 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:36.615916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08360e-ee67-4621-8f1e-cc1e173c3815 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:36.615916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd08360e-ee67-4621-8f1e-cc1e173c3815 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:36.618298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126176d6-dfbb-429f-9b98-97cf99d59d38 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:36.621561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 231/461] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:36.622598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08360e-ee67-4621-8f1e-cc1e173c3815 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:36.667776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16decd-2394-4e38-a0fe-1aea59845752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:36.668411 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16decd-2394-4e38-a0fe-1aea59845752 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:36.668686 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16decd-2394-4e38-a0fe-1aea59845752 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:36.668957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16decd-2394-4e38-a0fe-1aea59845752 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:36.686390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16decd-2394-4e38-a0fe-1aea59845752 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:36.688261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16decd-2394-4e38-a0fe-1aea59845752 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:36.709270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:36.717987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0c16decd-2394-4e38-a0fe-1aea59845752 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:36.717987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:36.717987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:36.717987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:36.717987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:36.717987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:36.717987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:36.717987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:36.717987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:36.717987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:36.717987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:36.717987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:36.717987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:36.717987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:36.717987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:36.717987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:36.719099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:36.719957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:36.719957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:36.719957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:36.719957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 231/462] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:36.720870 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5de98a-9667-47d8-bdff-596ce32b85f2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:36.720974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5de98a-9667-47d8-bdff-596ce32b85f2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:36.721195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5de98a-9667-47d8-bdff-596ce32b85f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:36.742485 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9a5de98a-9667-47d8-bdff-596ce32b85f2 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:36.744203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9a5de98a-9667-47d8-bdff-596ce32b85f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:36.751479 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd08360e-ee67-4621-8f1e-cc1e173c3815 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:36.797563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0c0db3-c72c-4694-8f18-838e9ed9f420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:36.798621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c0db3-c72c-4694-8f18-838e9ed9f420 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:36.799081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c0db3-c72c-4694-8f18-838e9ed9f420 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:36.799581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c0db3-c72c-4694-8f18-838e9ed9f420 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:36.800315 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c0db3-c72c-4694-8f18-838e9ed9f420 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:36.800622 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c0db3-c72c-4694-8f18-838e9ed9f420 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:36.802310 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0c0db3-c72c-4694-8f18-838e9ed9f420 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:36.822932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 232/463] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:36.822932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b8191-4da0-497a-8f82-849ef4411189 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:36.822932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b8191-4da0-497a-8f82-849ef4411189 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:36.822932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b8191-4da0-497a-8f82-849ef4411189 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:36.822932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 233/464] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:36.822932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b8191-4da0-497a-8f82-849ef4411189 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:36.832097 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 232/465] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:36.832778 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5de98a-9667-47d8-bdff-596ce32b85f2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:36.874610 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:36.972878 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b8191-4da0-497a-8f82-849ef4411189 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:36.973508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5de98a-9667-47d8-bdff-596ce32b85f2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:37.021093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:37.021667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb75df6-1f9a-4f59-a2cf-860b15286dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:37.022829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb75df6-1f9a-4f59-a2cf-860b15286dbd None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:37.023275 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb75df6-1f9a-4f59-a2cf-860b15286dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:37.023557 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb75df6-1f9a-4f59-a2cf-860b15286dbd None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:37.024607 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb75df6-1f9a-4f59-a2cf-860b15286dbd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:37.027214 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb75df6-1f9a-4f59-a2cf-860b15286dbd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:37.029384 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd9f09-e55a-499e-bac2-87bab28a22b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:37.030071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd9f09-e55a-499e-bac2-87bab28a22b5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:37.030382 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd9f09-e55a-499e-bac2-87bab28a22b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:37.030657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd9f09-e55a-499e-bac2-87bab28a22b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:37.031307 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd9f09-e55a-499e-bac2-87bab28a22b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:37.031604 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd9f09-e55a-499e-bac2-87bab28a22b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:37.032967 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd9f09-e55a-499e-bac2-87bab28a22b5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:37.049687 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 234/466] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:37.050994 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e0b17-98d7-44c7-8276-8fd9559ab9bf None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:37.051400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e0b17-98d7-44c7-8276-8fd9559ab9bf None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:37.051612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e0b17-98d7-44c7-8276-8fd9559ab9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:37.051712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 233/467] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:13:36 2022] POST /identity/v3/users => generated 246 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:13:37.067296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb75df6-1f9a-4f59-a2cf-860b15286dbd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:37.073591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c59e0b17-98d7-44c7-8276-8fd9559ab9bf None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:37.075032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c59e0b17-98d7-44c7-8276-8fd9559ab9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:37.187366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 235/468] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:37.188331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e0b17-98d7-44c7-8276-8fd9559ab9bf None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:37.219349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:37.243064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb75df6-1f9a-4f59-a2cf-860b15286dbd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:37.298974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333199a-4e49-41a0-a677-2ee194bb1373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:37.299662 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333199a-4e49-41a0-a677-2ee194bb1373 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:37.299944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333199a-4e49-41a0-a677-2ee194bb1373 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:37.300211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f333199a-4e49-41a0-a677-2ee194bb1373 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:37.300850 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333199a-4e49-41a0-a677-2ee194bb1373 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:37.302334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f333199a-4e49-41a0-a677-2ee194bb1373 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:37.319442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e0b17-98d7-44c7-8276-8fd9559ab9bf None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:37.326489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 234/469] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:13:37 2022] POST /identity/v3/users => generated 248 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:13:37.361060 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:37.366252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282085d9-3a91-4b61-81ab-8fa7920a1804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:37.366922 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282085d9-3a91-4b61-81ab-8fa7920a1804 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:37.367201 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282085d9-3a91-4b61-81ab-8fa7920a1804 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:37.367530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282085d9-3a91-4b61-81ab-8fa7920a1804 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:37.381071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282085d9-3a91-4b61-81ab-8fa7920a1804 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:37.382879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282085d9-3a91-4b61-81ab-8fa7920a1804 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:37.400434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-282085d9-3a91-4b61-81ab-8fa7920a1804 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:37.400434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:37.400434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:37.400434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:37.400434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:37.400434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:37.400434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:37.400434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:37.400434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:37.400434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:37.400434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:37.400434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:37.400434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:37.400434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:37.400434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:37.400434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:37.401662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:37.402554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:37.402554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:37.402554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:37.406730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 236/470] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:37.408885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333199a-4e49-41a0-a677-2ee194bb1373 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:37.527061 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282085d9-3a91-4b61-81ab-8fa7920a1804 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:37.532769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f333199a-4e49-41a0-a677-2ee194bb1373 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:37.589354 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a4880e-b416-4519-872c-c39eadce4307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:37.590388 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4880e-b416-4519-872c-c39eadce4307 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:37.590854 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4880e-b416-4519-872c-c39eadce4307 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:37.591880 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a4880e-b416-4519-872c-c39eadce4307 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:37.591880 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a4880e-b416-4519-872c-c39eadce4307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:37.592141 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a4880e-b416-4519-872c-c39eadce4307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:37.593434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a4880e-b416-4519-872c-c39eadce4307 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:37.609294 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 235/471] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:37.610617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a404836-0ae3-4b69-9958-8d77f28c2296 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:37.610904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a404836-0ae3-4b69-9958-8d77f28c2296 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:37.611178 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a404836-0ae3-4b69-9958-8d77f28c2296 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:37.612522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 236/472] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:37.615098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a404836-0ae3-4b69-9958-8d77f28c2296 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:37.620668 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:37.668699 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282085d9-3a91-4b61-81ab-8fa7920a1804 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:37.705224 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e385c56-1546-429d-a3e3-dc2bf90c7263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:37.705895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e385c56-1546-429d-a3e3-dc2bf90c7263 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:37.706174 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e385c56-1546-429d-a3e3-dc2bf90c7263 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:37.706489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e385c56-1546-429d-a3e3-dc2bf90c7263 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:37.720847 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e385c56-1546-429d-a3e3-dc2bf90c7263 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:37.723017 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e385c56-1546-429d-a3e3-dc2bf90c7263 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:37.741877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-4e385c56-1546-429d-a3e3-dc2bf90c7263 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:37.741877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:37.741877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:37.741877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:37.741877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:37.741877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:37.741877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:37.741877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:37.741877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:37.741877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:37.741877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:37.741877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:37.741877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:37.741877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:37.741877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:37.741877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:37.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:37.743674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:37.743674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:37.743674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:37.743674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 237/473] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:13:37.748936 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd4cdd-067c-49e6-bb2f-8bbb0c931292 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:37.748936 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd4cdd-067c-49e6-bb2f-8bbb0c931292 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:37.748936 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd4cdd-067c-49e6-bb2f-8bbb0c931292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:37.764565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a404836-0ae3-4b69-9958-8d77f28c2296 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:37.765861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d6fd4cdd-067c-49e6-bb2f-8bbb0c931292 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:37.767181 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d6fd4cdd-067c-49e6-bb2f-8bbb0c931292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:37.792156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:37.815943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf7d5e-c4c2-497c-87da-9159b6c86d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:37.816880 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf7d5e-c4c2-497c-87da-9159b6c86d17 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:37.817324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf7d5e-c4c2-497c-87da-9159b6c86d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:37.818754 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bf7d5e-c4c2-497c-87da-9159b6c86d17 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:37.821870 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf7d5e-c4c2-497c-87da-9159b6c86d17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:37.823437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bf7d5e-c4c2-497c-87da-9159b6c86d17 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:37.854595 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 237/474] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:13:37 2022] POST /identity/v3/users => generated 248 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:13:37.855093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 238/475] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:37.862930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf7d5e-c4c2-497c-87da-9159b6c86d17 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:37.965407 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:38.026403 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bf7d5e-c4c2-497c-87da-9159b6c86d17 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:38.074453 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc7ce8-a79c-472c-91a5-48bf575eeaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:38.075734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc7ce8-a79c-472c-91a5-48bf575eeaf5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:38.076214 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc7ce8-a79c-472c-91a5-48bf575eeaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:38.076837 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fc7ce8-a79c-472c-91a5-48bf575eeaf5 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:38.077724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc7ce8-a79c-472c-91a5-48bf575eeaf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:38.078490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc7ce8-a79c-472c-91a5-48bf575eeaf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:38.081023 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fc7ce8-a79c-472c-91a5-48bf575eeaf5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:38.108790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 238/476] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:38.117135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd4cdd-067c-49e6-bb2f-8bbb0c931292 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:38.227026 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc7ce8-a79c-472c-91a5-48bf575eeaf5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:38.236264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd4cdd-067c-49e6-bb2f-8bbb0c931292 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:38.272115 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1e048-2056-438e-b081-1bd321e5221e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:38.272780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1e048-2056-438e-b081-1bd321e5221e None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:38.273059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1e048-2056-438e-b081-1bd321e5221e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:38.273326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1e048-2056-438e-b081-1bd321e5221e None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:38.273966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1e048-2056-438e-b081-1bd321e5221e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:38.275491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1e048-2056-438e-b081-1bd321e5221e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:38.297254 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 239/477] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:13:38 2022] POST /identity/v3/users => generated 249 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:13:38.357989 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fc7ce8-a79c-472c-91a5-48bf575eeaf5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:38.384017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:38.405758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db116ec-c212-44c0-a5f2-6379beb9d1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:38.406724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db116ec-c212-44c0-a5f2-6379beb9d1cd None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:38.407151 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db116ec-c212-44c0-a5f2-6379beb9d1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:38.407623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db116ec-c212-44c0-a5f2-6379beb9d1cd None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:38.408530 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db116ec-c212-44c0-a5f2-6379beb9d1cd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:38.410133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db116ec-c212-44c0-a5f2-6379beb9d1cd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:38.419438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.sql.core [None req-3db116ec-c212-44c0-a5f2-6379beb9d1cd None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 15:13:38.419438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 15:13:38.419438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [parameters: {'id': '7df80e45cd4e4c78b0c4710d84e89592', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 15:13:38.419438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81620) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 15:13:38.440613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-3db116ec-c212-44c0-a5f2-6379beb9d1cd None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 15:13:38.440613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:38.440613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 15:13:38.440613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 15:13:38.440613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 15:13:38.440613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 15:13:38.440613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 15:13:38.440613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 15:13:38.440613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 15:13:38.440613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application conn.query(q) Aug 24 15:13:38.440613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 15:13:38.440613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 15:13:38.440613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 15:13:38.440613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result.read() Aug 24 15:13:38.440613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 15:13:38.441828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application next(self.gen) Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application yield resource Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application next(self.gen) Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise self.value Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application session.commit() Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 15:13:38.444720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.session.flush() Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application compat.raise_( Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 15:13:38.446931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 15:13:38.449053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 15:13:38.449053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 15:13:38.449053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 15:13:38.449053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 15:13:38.449053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 15:13:38.449053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 15:13:38.449053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 15:13:38.449053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 15:13:38.449053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 15:13:38.449053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 15:13:38.449053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 15:13:38.449053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 15:13:38.449053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 15:13:38.449053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 15:13:38.449053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 15:13:38.449053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception Aug 24 15:13:38.452329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 15:13:38.452329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 15:13:38.452329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 15:13:38.452329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 15:13:38.452329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 15:13:38.452329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 15:13:38.452329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 15:13:38.452329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application conn.query(q) Aug 24 15:13:38.452329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 15:13:38.452329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 15:13:38.452329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 15:13:38.452329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result.read() Aug 24 15:13:38.452329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 15:13:38.452329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 15:13:38.452329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 15:13:38.452329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 15:13:38.457258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 15:13:38.457258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 15:13:38.457258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 15:13:38.457258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 15:13:38.457258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 15:13:38.457258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 15:13:38.457258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [parameters: {'id': '7df80e45cd4e4c78b0c4710d84e89592', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 15:13:38.457258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 15:13:38.457258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:38.457258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:13:38.457258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:38.457258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:38.457258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:38.457258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:38.457258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:38.457258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:38.463345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:38.463345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:38.463345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:38.463345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:38.463345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:38.463345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:38.463345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 15:13:38.463345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 15:13:38.463345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:38.463345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:38.463345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 15:13:38.463345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 15:13:38.463345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 15:13:38.463345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 15:13:38.463345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 15:13:38.463345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:38.465664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 239/478] 10.222.0.76 () {60 vars in 1131 bytes} [Wed Aug 24 15:13:38 2022] POST /identity/v3/roles => generated 144 bytes in 235 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:13:38.471351 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1e048-2056-438e-b081-1bd321e5221e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:38.509069 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:38.608632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1e048-2056-438e-b081-1bd321e5221e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:38.647180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c432937-9726-4944-9877-e02cba44a3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:38.648163 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c432937-9726-4944-9877-e02cba44a3ee None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:38.648591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c432937-9726-4944-9877-e02cba44a3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:38.649026 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c432937-9726-4944-9877-e02cba44a3ee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:38.664081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c432937-9726-4944-9877-e02cba44a3ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:38.670564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-5c432937-9726-4944-9877-e02cba44a3ee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:38.670564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:38.670564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:38.670564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:38.670564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:38.670564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:38.670564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:38.670564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:38.670564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:38.670564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:38.670564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:38.670564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:38.670564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:38.670564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:38.670564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:38.670564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:38.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:38.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:38.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:38.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:38.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:38.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:38.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:38.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:38.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:38.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:38.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:38.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:38.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:38.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:38.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:38.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:38.672259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:38.672259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:38.672259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:38.672259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:38.672259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:38.672259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:38.672259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:38.672259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:38.672259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:38.672259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:38.672259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 15:13:38.672259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:38.672259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 240/479] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:13:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:38.679278 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db116ec-c212-44c0-a5f2-6379beb9d1cd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:38.826579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db116ec-c212-44c0-a5f2-6379beb9d1cd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:38.877314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf0e49-9681-40b9-8531-3c09d8a3cf37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:38.878324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf0e49-9681-40b9-8531-3c09d8a3cf37 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:38.878806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf0e49-9681-40b9-8531-3c09d8a3cf37 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:38.879261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf0e49-9681-40b9-8531-3c09d8a3cf37 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:38.880130 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf0e49-9681-40b9-8531-3c09d8a3cf37 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:38.881162 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf0e49-9681-40b9-8531-3c09d8a3cf37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:38.882745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf0e49-9681-40b9-8531-3c09d8a3cf37 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:38.892443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 240/480] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:39.626121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d1f9c-db62-4ab6-b53b-03a958899407 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:39.627102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d1f9c-db62-4ab6-b53b-03a958899407 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:39.627594 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d1f9c-db62-4ab6-b53b-03a958899407 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:39.629431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 241/481] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:39.634755 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf57a8-79c8-4ed0-ad1e-8d35c10ca6e8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:39.635214 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf57a8-79c8-4ed0-ad1e-8d35c10ca6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:39.635315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf57a8-79c8-4ed0-ad1e-8d35c10ca6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:39.659315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4acf57a8-79c8-4ed0-ad1e-8d35c10ca6e8 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:39.660553 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4acf57a8-79c8-4ed0-ad1e-8d35c10ca6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:39.727310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:39.768430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 241/482] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:13:39.813064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c69344-2908-459d-b001-4c3563df59a8 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:39.813383 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c69344-2908-459d-b001-4c3563df59a8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:39.813556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c69344-2908-459d-b001-4c3563df59a8 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:39.815116 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 242/483] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:39.821077 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68cd59-3c90-4629-8106-658efd0e88da None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:39.821369 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68cd59-3c90-4629-8106-658efd0e88da None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:39.821649 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee68cd59-3c90-4629-8106-658efd0e88da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:39.851207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ee68cd59-3c90-4629-8106-658efd0e88da None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:39.852992 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ee68cd59-3c90-4629-8106-658efd0e88da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:39.947869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 242/484] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:40.139039 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c69344-2908-459d-b001-4c3563df59a8 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:40.260071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:40.308969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c69344-2908-459d-b001-4c3563df59a8 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:40.349927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68cd59-3c90-4629-8106-658efd0e88da None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:40.366197 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a8959-5187-4e75-85d6-d1558535f9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:40.368306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a8959-5187-4e75-85d6-d1558535f9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:40.368582 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a8959-5187-4e75-85d6-d1558535f9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:40.368853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a8959-5187-4e75-85d6-d1558535f9f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:40.390560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a8959-5187-4e75-85d6-d1558535f9f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:40.397921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-0c4a8959-5187-4e75-85d6-d1558535f9f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:40.397921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:40.397921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:40.397921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:40.397921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:40.397921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:40.397921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:40.397921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:40.397921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:40.397921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:40.397921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:40.397921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:40.397921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:40.397921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:40.397921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:40.397921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:40.398916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:40.398916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:40.398916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:40.398916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:40.398916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:40.398916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:40.398916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:40.398916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:40.398916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:40.398916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:40.398916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:40.398916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:40.398916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:40.398916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:40.398916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:40.398916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:40.399778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:40.399778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:40.399778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:40.399778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:40.399778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:40.399778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:40.399778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:40.399778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:40.399778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:40.399778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:40.399778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:40.399778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:40.400372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 243/485] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:40.407521 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a8959-5187-4e75-85d6-d1558535f9f0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:40.563026 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee68cd59-3c90-4629-8106-658efd0e88da None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:40.575444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:40.606826 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c84f55-4dd1-4ad7-b844-4a5ce9f09328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:40.607760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c84f55-4dd1-4ad7-b844-4a5ce9f09328 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:40.608199 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c84f55-4dd1-4ad7-b844-4a5ce9f09328 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:40.608622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c84f55-4dd1-4ad7-b844-4a5ce9f09328 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:40.616759 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a8959-5187-4e75-85d6-d1558535f9f0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:40.621760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c84f55-4dd1-4ad7-b844-4a5ce9f09328 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:40.627923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-53c84f55-4dd1-4ad7-b844-4a5ce9f09328 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:40.627923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:40.627923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:40.627923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:40.627923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:40.627923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:40.627923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:40.627923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:40.627923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:40.627923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:40.627923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:40.627923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:40.627923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:40.627923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:40.627923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:40.627923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:40.628885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:40.628885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:40.628885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:40.628885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:40.628885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:40.628885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:40.628885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:40.628885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:40.628885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:40.628885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:40.628885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:40.628885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:40.628885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:40.628885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:40.628885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:40.628885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:40.629758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:40.629758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:40.629758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:40.629758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:40.629758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:40.629758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:40.629758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:40.629758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:40.629758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:40.629758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:40.629758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:40.629758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:40.631990 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 243/486] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:13:40.641288 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c84f55-4dd1-4ad7-b844-4a5ce9f09328 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:40.666851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:40.675018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca60d3-d419-407d-878b-565c9dd2d633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:40.676029 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca60d3-d419-407d-878b-565c9dd2d633 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:40.676800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca60d3-d419-407d-878b-565c9dd2d633 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:40.677703 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca60d3-d419-407d-878b-565c9dd2d633 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:40.678596 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca60d3-d419-407d-878b-565c9dd2d633 None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:40.679080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca60d3-d419-407d-878b-565c9dd2d633 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:40.681441 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca60d3-d419-407d-878b-565c9dd2d633 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:40.692158 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 244/487] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:13:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:40.697372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd0ee7-1a3f-4b66-961a-e0c382b64bcc None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:40.697372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd0ee7-1a3f-4b66-961a-e0c382b64bcc None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:40.697372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd0ee7-1a3f-4b66-961a-e0c382b64bcc None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:40.699206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 245/488] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:40.700061 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd0ee7-1a3f-4b66-961a-e0c382b64bcc None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:40.781294 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c84f55-4dd1-4ad7-b844-4a5ce9f09328 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:40.838466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25b8bd-2cc6-4d88-b2f9-a73c7df92212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:40.839430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25b8bd-2cc6-4d88-b2f9-a73c7df92212 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:40.839897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25b8bd-2cc6-4d88-b2f9-a73c7df92212 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:40.840359 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25b8bd-2cc6-4d88-b2f9-a73c7df92212 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:40.841113 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25b8bd-2cc6-4d88-b2f9-a73c7df92212 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:40.841513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25b8bd-2cc6-4d88-b2f9-a73c7df92212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:40.843302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25b8bd-2cc6-4d88-b2f9-a73c7df92212 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:40.843934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd0ee7-1a3f-4b66-961a-e0c382b64bcc None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:40.862738 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 244/489] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:13:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:40.864445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca772625-cd92-43bb-9879-8a7013788556 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:40.864876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca772625-cd92-43bb-9879-8a7013788556 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:40.865247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca772625-cd92-43bb-9879-8a7013788556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:40.882298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce732a2e-efe3-4ffa-ab3e-199e79d2bceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:40.882930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce732a2e-efe3-4ffa-ab3e-199e79d2bceb None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:40.883201 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce732a2e-efe3-4ffa-ab3e-199e79d2bceb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:40.883469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce732a2e-efe3-4ffa-ab3e-199e79d2bceb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:40.885315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ca772625-cd92-43bb-9879-8a7013788556 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:40.886759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ca772625-cd92-43bb-9879-8a7013788556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:40.894485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce732a2e-efe3-4ffa-ab3e-199e79d2bceb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:40.895974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce732a2e-efe3-4ffa-ab3e-199e79d2bceb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:40.910970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ce732a2e-efe3-4ffa-ab3e-199e79d2bceb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:40.910970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:40.910970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:40.910970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:40.910970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:40.910970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:40.910970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:40.910970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:40.910970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:40.910970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:40.910970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:40.910970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:40.910970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:40.910970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:40.910970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:40.910970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:40.912127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:40.913626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:40.913626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:40.913626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:40.913626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 246/490] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:40.913626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce732a2e-efe3-4ffa-ab3e-199e79d2bceb None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:40.919178 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:40.975940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 245/491] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:40.978900 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca772625-cd92-43bb-9879-8a7013788556 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:40.993271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:41.099692 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce732a2e-efe3-4ffa-ab3e-199e79d2bceb None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:41.152133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3b005e-677a-4706-b077-85ad96341a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:41.152689 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b005e-677a-4706-b077-85ad96341a7a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:41.152968 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b005e-677a-4706-b077-85ad96341a7a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:41.153240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3b005e-677a-4706-b077-85ad96341a7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:41.165175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca772625-cd92-43bb-9879-8a7013788556 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:41.165399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b005e-677a-4706-b077-85ad96341a7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:41.166854 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3b005e-677a-4706-b077-85ad96341a7a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:41.181317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ef3b005e-677a-4706-b077-85ad96341a7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:41.181317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:41.181317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:41.181317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:41.181317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:41.181317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:41.181317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:41.181317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:41.181317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:41.181317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:41.181317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:41.181317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:41.181317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:41.181317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:41.181317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:41.181317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:41.182282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:41.183117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:41.183117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:41.183117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:41.183117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 247/492] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:41.183906 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417969f7-9005-4a1c-9d51-30c8aac7aabc None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:41.184186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417969f7-9005-4a1c-9d51-30c8aac7aabc None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:41.184456 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417969f7-9005-4a1c-9d51-30c8aac7aabc None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:41.185689 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 248/493] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:41.190621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417969f7-9005-4a1c-9d51-30c8aac7aabc None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:41.213326 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd04c4-1188-46f7-99bc-9c6563e85b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:41.214211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd04c4-1188-46f7-99bc-9c6563e85b55 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:41.214749 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd04c4-1188-46f7-99bc-9c6563e85b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:41.215211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd04c4-1188-46f7-99bc-9c6563e85b55 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:41.216022 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd04c4-1188-46f7-99bc-9c6563e85b55 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:41.216459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd04c4-1188-46f7-99bc-9c6563e85b55 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:41.218075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bd04c4-1188-46f7-99bc-9c6563e85b55 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:41.231391 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 246/494] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:41.233058 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f89d24-697c-48ba-885d-c294d1bb2262 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:41.233558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f89d24-697c-48ba-885d-c294d1bb2262 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:41.234021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f89d24-697c-48ba-885d-c294d1bb2262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:41.259839 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-37f89d24-697c-48ba-885d-c294d1bb2262 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:41.261438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-37f89d24-697c-48ba-885d-c294d1bb2262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:41.276286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:41.297933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:41.305377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417969f7-9005-4a1c-9d51-30c8aac7aabc None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:41.343331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa7b89-ffa0-476f-b5dc-0103b7b537cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:41.350292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 247/495] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:41.350441 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa7b89-ffa0-476f-b5dc-0103b7b537cf None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:41.350441 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa7b89-ffa0-476f-b5dc-0103b7b537cf None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:41.350441 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fa7b89-ffa0-476f-b5dc-0103b7b537cf None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:41.350673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f89d24-697c-48ba-885d-c294d1bb2262 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:41.350746 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa7b89-ffa0-476f-b5dc-0103b7b537cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:41.350746 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa7b89-ffa0-476f-b5dc-0103b7b537cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:41.350746 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fa7b89-ffa0-476f-b5dc-0103b7b537cf None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:41.372862 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 249/496] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:41.372862 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa7b89-ffa0-476f-b5dc-0103b7b537cf None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:41.479316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fa7b89-ffa0-476f-b5dc-0103b7b537cf None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:41.479643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f89d24-697c-48ba-885d-c294d1bb2262 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:41.517962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8af3d6-82eb-4924-9d13-494f70a4bf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:41.518622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8af3d6-82eb-4924-9d13-494f70a4bf65 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:41.518900 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8af3d6-82eb-4924-9d13-494f70a4bf65 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:41.519209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8af3d6-82eb-4924-9d13-494f70a4bf65 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:41.527767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8af3d6-82eb-4924-9d13-494f70a4bf65 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:41.529302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8af3d6-82eb-4924-9d13-494f70a4bf65 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:41.535785 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d379d-c244-4513-b533-15ed7b6cea7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:41.537334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d379d-c244-4513-b533-15ed7b6cea7b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:41.537804 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d379d-c244-4513-b533-15ed7b6cea7b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:41.538474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d379d-c244-4513-b533-15ed7b6cea7b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:41.545161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5a8af3d6-82eb-4924-9d13-494f70a4bf65 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 15:13:41.545161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:41.545161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:41.545161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:41.545161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:41.545161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:41.545161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:41.545161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:41.545161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:41.545161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:41.545161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:41.545161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:41.545161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:13:41.545161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:13:41.545161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:13:41.545161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:41.546142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:41.547035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:41.547035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:41.547035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:13:41.547035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:13:41.547035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:13:41.547035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:13:41.547035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:13:41.547035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:13:41.547035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 15:13:41.547035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:41.550970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 248/497] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:41.552016 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8af3d6-82eb-4924-9d13-494f70a4bf65 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:41.558001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d379d-c244-4513-b533-15ed7b6cea7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:41.563635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-c58d379d-c244-4513-b533-15ed7b6cea7b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:41.563635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:41.563635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:41.563635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:41.563635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:41.563635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:41.563635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:41.563635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:41.563635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:41.563635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:41.563635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:41.563635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:41.563635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:41.563635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:41.563635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:41.563635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:41.564560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:41.564560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:41.564560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:41.564560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:41.564560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:41.564560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:41.564560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:41.564560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:41.564560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:41.564560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:41.564560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:41.564560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:41.564560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:41.564560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:41.564560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:41.564560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:41.565361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:41.565361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:41.565361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:41.565361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:41.565361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:41.565361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:41.565361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:41.565361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:41.565361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:41.565361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:41.565361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:41.565361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:41.566037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:41.568203 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 250/498] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:41.569212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d379d-c244-4513-b533-15ed7b6cea7b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:41.596896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:41.670635 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8af3d6-82eb-4924-9d13-494f70a4bf65 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:41.702366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d379d-c244-4513-b533-15ed7b6cea7b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:41.708186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ba40c-7992-4fb4-b1d0-b9cf359a7183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:41.708815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ba40c-7992-4fb4-b1d0-b9cf359a7183 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:41.709091 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ba40c-7992-4fb4-b1d0-b9cf359a7183 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:41.709418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244ba40c-7992-4fb4-b1d0-b9cf359a7183 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:41.717588 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ba40c-7992-4fb4-b1d0-b9cf359a7183 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:41.719057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244ba40c-7992-4fb4-b1d0-b9cf359a7183 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:41.730164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-244ba40c-7992-4fb4-b1d0-b9cf359a7183 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:13:41.730164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:41.730164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:41.730164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:41.730164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:41.730164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:41.730164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:41.730164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:41.730164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:41.730164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:41.730164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:41.730164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:41.730164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:13:41.730164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:13:41.730164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:13:41.730164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:41.731093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:41.731988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:41.731988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:41.731988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:13:41.731988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:13:41.731988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:13:41.731988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:13:41.731988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:13:41.731988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:13:41.731988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:13:41.731988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:41.731988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 249/499] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 15:13:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:41.731988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ba40c-7992-4fb4-b1d0-b9cf359a7183 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:41.744991 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ee1f8-91c7-497f-9d62-5c517f62767c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:41.746777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ee1f8-91c7-497f-9d62-5c517f62767c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:41.746777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ee1f8-91c7-497f-9d62-5c517f62767c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:41.746777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ee1f8-91c7-497f-9d62-5c517f62767c None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:41.747606 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ee1f8-91c7-497f-9d62-5c517f62767c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:41.747893 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ee1f8-91c7-497f-9d62-5c517f62767c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:41.749526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ee1f8-91c7-497f-9d62-5c517f62767c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:41.795020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 251/500] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 24 15:13:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:41.795020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9cc72-b84b-4d12-8f68-6cc2bee7f8b4 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:41.795020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9cc72-b84b-4d12-8f68-6cc2bee7f8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:41.795020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9cc72-b84b-4d12-8f68-6cc2bee7f8b4 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:41.795020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 252/501] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:41.795020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9cc72-b84b-4d12-8f68-6cc2bee7f8b4 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:41.805034 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:41.842311 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244ba40c-7992-4fb4-b1d0-b9cf359a7183 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:41.877453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:41.881081 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a9b3e-563d-468b-bcfd-e2924180abcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:41.881689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a9b3e-563d-468b-bcfd-e2924180abcd None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:41.881969 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a9b3e-563d-468b-bcfd-e2924180abcd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:41.882267 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a9b3e-563d-468b-bcfd-e2924180abcd None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:41.882932 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a9b3e-563d-468b-bcfd-e2924180abcd None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:41.883235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a9b3e-563d-468b-bcfd-e2924180abcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:41.884597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a9b3e-563d-468b-bcfd-e2924180abcd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:41.892683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 250/502] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:13:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:41.893471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a9b3e-563d-468b-bcfd-e2924180abcd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:41.903038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9cc72-b84b-4d12-8f68-6cc2bee7f8b4 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:41.938847 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd875a12-d29f-4803-baac-0b9268c8e8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:41.939460 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd875a12-d29f-4803-baac-0b9268c8e8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:41.939730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd875a12-d29f-4803-baac-0b9268c8e8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:41.939994 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd875a12-d29f-4803-baac-0b9268c8e8b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:41.950439 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd875a12-d29f-4803-baac-0b9268c8e8b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:41.955092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-cd875a12-d29f-4803-baac-0b9268c8e8b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:41.955092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:41.955092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:41.955092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:41.955092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:41.955092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:41.955092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:41.955092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:41.955092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:41.955092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:41.955092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:41.955092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:41.955092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:41.955092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:41.955092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:41.955092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:41.955997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:41.955997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:41.955997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:41.955997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:41.955997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:41.955997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:41.955997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:41.955997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:41.955997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:41.955997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:41.955997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:41.955997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:41.955997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:41.955997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:41.955997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:41.955997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:41.956773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:41.956773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:41.956773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:41.956773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:41.956773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:41.956773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:41.956773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:41.956773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:41.956773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:41.956773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:41.956773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:41.956773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:41.956773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 253/503] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:41.957645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd875a12-d29f-4803-baac-0b9268c8e8b2 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:42.003573 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a9b3e-563d-468b-bcfd-e2924180abcd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:42.033215 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:42.040252 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da18f7-b5c7-4b8a-b5c8-3727003aff9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:42.040865 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da18f7-b5c7-4b8a-b5c8-3727003aff9c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:42.041143 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da18f7-b5c7-4b8a-b5c8-3727003aff9c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:42.041415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da18f7-b5c7-4b8a-b5c8-3727003aff9c None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:42.042148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da18f7-b5c7-4b8a-b5c8-3727003aff9c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:42.042483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da18f7-b5c7-4b8a-b5c8-3727003aff9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:42.043914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9da18f7-b5c7-4b8a-b5c8-3727003aff9c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:42.060397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd875a12-d29f-4803-baac-0b9268c8e8b2 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:42.080293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 251/504] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 15:13:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:42.081515 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbaaf3-8b7c-413f-ae7e-7bc54c2d52b1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:42.081799 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbaaf3-8b7c-413f-ae7e-7bc54c2d52b1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:42.082074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dbaaf3-8b7c-413f-ae7e-7bc54c2d52b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:42.094420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcafca-cdbb-4699-b3ce-45a0a5042ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:42.095028 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcafca-cdbb-4699-b3ce-45a0a5042ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:42.095301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcafca-cdbb-4699-b3ce-45a0a5042ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:42.095571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdcafca-cdbb-4699-b3ce-45a0a5042ad0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:42.099132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:42.101475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c6dbaaf3-8b7c-413f-ae7e-7bc54c2d52b1 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:42.102817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c6dbaaf3-8b7c-413f-ae7e-7bc54c2d52b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:42.170733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 252/505] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:42.171664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbaaf3-8b7c-413f-ae7e-7bc54c2d52b1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:42.183732 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcafca-cdbb-4699-b3ce-45a0a5042ad0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:42.185463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdcafca-cdbb-4699-b3ce-45a0a5042ad0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:42.282823 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dbaaf3-8b7c-413f-ae7e-7bc54c2d52b1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:42.303710 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 254/506] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:42.304634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcafca-cdbb-4699-b3ce-45a0a5042ad0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:42.308556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:42.329461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599c69c-b05a-4ba8-b035-9223ee71a94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:42.330210 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599c69c-b05a-4ba8-b035-9223ee71a94a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:42.330519 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599c69c-b05a-4ba8-b035-9223ee71a94a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:42.330730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2599c69c-b05a-4ba8-b035-9223ee71a94a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:42.342122 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599c69c-b05a-4ba8-b035-9223ee71a94a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:42.344029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2599c69c-b05a-4ba8-b035-9223ee71a94a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:42.345719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:42.361434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2599c69c-b05a-4ba8-b035-9223ee71a94a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:42.361434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:42.361434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:42.361434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:42.361434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:42.361434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:42.361434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:42.361434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:42.361434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:42.361434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:42.361434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:42.361434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:42.361434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:42.361434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:42.361434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:42.361434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:42.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:42.363590 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:42.363590 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:42.363590 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:42.367568 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 253/507] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:42.368560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599c69c-b05a-4ba8-b035-9223ee71a94a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:42.429814 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdcafca-cdbb-4699-b3ce-45a0a5042ad0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:42.468499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd663793-b650-49e4-9b76-57a421594475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:42.469544 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd663793-b650-49e4-9b76-57a421594475 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:42.469855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd663793-b650-49e4-9b76-57a421594475 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:42.470063 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd663793-b650-49e4-9b76-57a421594475 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:42.470752 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd663793-b650-49e4-9b76-57a421594475 None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:42.471052 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd663793-b650-49e4-9b76-57a421594475 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:42.472511 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd663793-b650-49e4-9b76-57a421594475 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:42.481037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 255/508] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:13:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:42.481860 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd663793-b650-49e4-9b76-57a421594475 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:42.495158 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2599c69c-b05a-4ba8-b035-9223ee71a94a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:42.544467 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83cbd3-c668-4331-8bc5-313bddc1bff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:42.545093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83cbd3-c668-4331-8bc5-313bddc1bff5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:42.545461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83cbd3-c668-4331-8bc5-313bddc1bff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:42.545946 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae83cbd3-c668-4331-8bc5-313bddc1bff5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:42.596312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:42.611439 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:42.617316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd663793-b650-49e4-9b76-57a421594475 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:42.631506 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83cbd3-c668-4331-8bc5-313bddc1bff5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:42.632971 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae83cbd3-c668-4331-8bc5-313bddc1bff5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:42.653329 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19c868-71d8-46cb-a3da-375d0ed9a7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:42.654037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19c868-71d8-46cb-a3da-375d0ed9a7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:42.654343 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19c868-71d8-46cb-a3da-375d0ed9a7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:42.654614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19c868-71d8-46cb-a3da-375d0ed9a7d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:42.665834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19c868-71d8-46cb-a3da-375d0ed9a7d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:42.667338 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19c868-71d8-46cb-a3da-375d0ed9a7d0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:42.682021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-2d19c868-71d8-46cb-a3da-375d0ed9a7d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:42.682021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:42.682021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:42.682021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:42.682021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:42.682021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:42.682021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:42.682021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:42.682021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:42.682021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:42.682021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:42.682021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:42.682021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:42.682021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:42.682021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:42.682021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:42.682989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:42.683832 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:42.683832 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:42.683832 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:42.683832 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 256/509] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:42.684040 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19c868-71d8-46cb-a3da-375d0ed9a7d0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:42.751849 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 254/510] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:42.752706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83cbd3-c668-4331-8bc5-313bddc1bff5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:42.798556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19c868-71d8-46cb-a3da-375d0ed9a7d0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:42.830470 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:42.838617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514ecf0-62a3-41b6-a9cb-dc433c109c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:42.839221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514ecf0-62a3-41b6-a9cb-dc433c109c1f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:42.839495 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514ecf0-62a3-41b6-a9cb-dc433c109c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:42.839762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514ecf0-62a3-41b6-a9cb-dc433c109c1f None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:42.840402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514ecf0-62a3-41b6-a9cb-dc433c109c1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:42.840699 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514ecf0-62a3-41b6-a9cb-dc433c109c1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:42.842084 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514ecf0-62a3-41b6-a9cb-dc433c109c1f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:42.855047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 257/511] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:42.855998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514ecf0-62a3-41b6-a9cb-dc433c109c1f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:42.872559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:42.880203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae83cbd3-c668-4331-8bc5-313bddc1bff5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:42.924096 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7f4e0-39ba-46d2-822b-5901a297eb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:42.924684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7f4e0-39ba-46d2-822b-5901a297eb0c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:42.924962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7f4e0-39ba-46d2-822b-5901a297eb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:42.925227 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a7f4e0-39ba-46d2-822b-5901a297eb0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:42.940786 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7f4e0-39ba-46d2-822b-5901a297eb0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:42.943725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a7f4e0-39ba-46d2-822b-5901a297eb0c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:42.962095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f0a7f4e0-39ba-46d2-822b-5901a297eb0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:42.962095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:42.962095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:42.962095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:42.962095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:42.962095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:42.962095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:42.962095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:42.962095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:42.962095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:42.962095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:42.962095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:42.962095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:42.962095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:42.962095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:42.962095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:42.963835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:42.965679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:42.965679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:42.965679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:42.965679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 255/512] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:42.965679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7f4e0-39ba-46d2-822b-5901a297eb0c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:42.975667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514ecf0-62a3-41b6-a9cb-dc433c109c1f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:43.022837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28716d4e-8699-4aeb-a665-f188a4154a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:43.023467 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28716d4e-8699-4aeb-a665-f188a4154a21 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:43.023743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28716d4e-8699-4aeb-a665-f188a4154a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:43.024011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28716d4e-8699-4aeb-a665-f188a4154a21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:43.038862 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28716d4e-8699-4aeb-a665-f188a4154a21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:43.038862 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28716d4e-8699-4aeb-a665-f188a4154a21 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:43.055081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-28716d4e-8699-4aeb-a665-f188a4154a21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:43.055081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:43.055081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:43.055081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:43.055081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:43.055081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:43.055081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:43.055081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:43.055081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:43.055081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:43.055081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:43.055081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:43.055081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:43.055081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:43.055081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:43.055081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:43.057008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:43.059010 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:43.059010 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:43.059010 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:43.059010 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 258/513] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:43.059010 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28716d4e-8699-4aeb-a665-f188a4154a21 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:43.093678 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a7f4e0-39ba-46d2-822b-5901a297eb0c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:43.121307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:43.128359 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:43.134816 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371b149-dab7-4706-8a3d-48ec0a103be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:43.135428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371b149-dab7-4706-8a3d-48ec0a103be9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:43.135708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371b149-dab7-4706-8a3d-48ec0a103be9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:43.135979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e371b149-dab7-4706-8a3d-48ec0a103be9 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:43.136793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371b149-dab7-4706-8a3d-48ec0a103be9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:43.136971 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371b149-dab7-4706-8a3d-48ec0a103be9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:43.138394 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e371b149-dab7-4706-8a3d-48ec0a103be9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:43.153170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 256/514] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:43.154038 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371b149-dab7-4706-8a3d-48ec0a103be9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:43.175287 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28716d4e-8699-4aeb-a665-f188a4154a21 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:43.211522 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394f76b-faff-4ad1-9055-40b8ddb25b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:43.212153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394f76b-faff-4ad1-9055-40b8ddb25b1b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:43.212431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394f76b-faff-4ad1-9055-40b8ddb25b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:43.212700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394f76b-faff-4ad1-9055-40b8ddb25b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:43.224663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394f76b-faff-4ad1-9055-40b8ddb25b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:43.226126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e394f76b-faff-4ad1-9055-40b8ddb25b1b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:43.241632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-e394f76b-faff-4ad1-9055-40b8ddb25b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:43.241632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:43.241632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:43.241632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:43.241632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:43.241632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:43.241632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:43.241632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:43.241632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:43.241632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:43.241632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:43.241632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:43.241632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:43.241632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:43.241632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:43.241632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:43.242790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:43.243820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:43.243820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:43.243820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:43.243820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 259/515] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:43.244126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394f76b-faff-4ad1-9055-40b8ddb25b1b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:43.270502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e371b149-dab7-4706-8a3d-48ec0a103be9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:43.323074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282c8fb-217c-4853-bb4a-18767a1a6b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:43.323730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c8fb-217c-4853-bb4a-18767a1a6b30 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:43.324181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c8fb-217c-4853-bb4a-18767a1a6b30 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:43.324535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9282c8fb-217c-4853-bb4a-18767a1a6b30 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:43.332743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282c8fb-217c-4853-bb4a-18767a1a6b30 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:43.334339 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9282c8fb-217c-4853-bb4a-18767a1a6b30 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:43.346338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-9282c8fb-217c-4853-bb4a-18767a1a6b30 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 15:13:43.346338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:43.346338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:43.346338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:43.346338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:43.346338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:43.346338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:43.346338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:43.346338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:43.346338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:43.346338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:43.346338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:43.346338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:13:43.346338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:13:43.346338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:13:43.346338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:43.348258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:43.350005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:43.350005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:43.350005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:13:43.350005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:13:43.350005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:13:43.350005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:13:43.350005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:13:43.350005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:13:43.350005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 15:13:43.350005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:43.350922 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e394f76b-faff-4ad1-9055-40b8ddb25b1b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:43.352797 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 257/516] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:43.353688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282c8fb-217c-4853-bb4a-18767a1a6b30 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:43.366269 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:43.388718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e48f7-7842-4438-8040-0fecef3f25b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:43.389322 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e48f7-7842-4438-8040-0fecef3f25b6 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:43.389595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e48f7-7842-4438-8040-0fecef3f25b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:43.389864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96e48f7-7842-4438-8040-0fecef3f25b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:43.390535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e48f7-7842-4438-8040-0fecef3f25b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:43.390832 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e48f7-7842-4438-8040-0fecef3f25b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:43.392216 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96e48f7-7842-4438-8040-0fecef3f25b6 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:43.399401 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:43.404840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 260/517] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:43.405665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e48f7-7842-4438-8040-0fecef3f25b6 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:43.474026 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9282c8fb-217c-4853-bb4a-18767a1a6b30 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:43.509100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96e48f7-7842-4438-8040-0fecef3f25b6 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:43.513620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89106e-4f05-42a2-942a-2f2c61a5c04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:43.514263 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89106e-4f05-42a2-942a-2f2c61a5c04f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:43.514549 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89106e-4f05-42a2-942a-2f2c61a5c04f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:43.514825 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d89106e-4f05-42a2-942a-2f2c61a5c04f None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:43.515476 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89106e-4f05-42a2-942a-2f2c61a5c04f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:43.515778 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89106e-4f05-42a2-942a-2f2c61a5c04f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:43.517151 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d89106e-4f05-42a2-942a-2f2c61a5c04f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:43.532840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 258/518] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:43.533691 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89106e-4f05-42a2-942a-2f2c61a5c04f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:43.547320 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd124f-f56a-4813-9099-8bd09f083edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:43.547953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd124f-f56a-4813-9099-8bd09f083edc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:43.548240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd124f-f56a-4813-9099-8bd09f083edc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:43.548518 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fd124f-f56a-4813-9099-8bd09f083edc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:43.555263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd124f-f56a-4813-9099-8bd09f083edc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:43.556715 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fd124f-f56a-4813-9099-8bd09f083edc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:43.568491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-21fd124f-f56a-4813-9099-8bd09f083edc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:43.568491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:43.568491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:43.568491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:43.568491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:43.568491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:43.568491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:43.568491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:43.568491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:43.568491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:43.568491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:43.568491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:43.568491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:13:43.568491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:13:43.568491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:43.569534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:43.570415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:13:43.570415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:13:43.570415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:13:43.570415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:13:43.570415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:13:43.570415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:13:43.570415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:43.570415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:43.570415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 261/519] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:43.570992 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd124f-f56a-4813-9099-8bd09f083edc None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:43.605292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:43.636810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d89106e-4f05-42a2-942a-2f2c61a5c04f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:43.638386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:43.673379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094045e-fcc7-4fc1-ad8e-61f07359bc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:43.673569 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fd124f-f56a-4813-9099-8bd09f083edc None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:43.673983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094045e-fcc7-4fc1-ad8e-61f07359bc23 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:43.674289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094045e-fcc7-4fc1-ad8e-61f07359bc23 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:43.674566 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094045e-fcc7-4fc1-ad8e-61f07359bc23 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:43.675210 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8094045e-fcc7-4fc1-ad8e-61f07359bc23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:43.675515 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8094045e-fcc7-4fc1-ad8e-61f07359bc23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:43.676878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8094045e-fcc7-4fc1-ad8e-61f07359bc23 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:43.688864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 259/520] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:43.689728 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094045e-fcc7-4fc1-ad8e-61f07359bc23 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:43.708203 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156026b-ea4d-42bc-9be1-9fb538990f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:43.708828 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156026b-ea4d-42bc-9be1-9fb538990f2a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:43.709101 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156026b-ea4d-42bc-9be1-9fb538990f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:43.709366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5156026b-ea4d-42bc-9be1-9fb538990f2a None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:43.710082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156026b-ea4d-42bc-9be1-9fb538990f2a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:43.710412 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156026b-ea4d-42bc-9be1-9fb538990f2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:43.711828 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5156026b-ea4d-42bc-9be1-9fb538990f2a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:43.729829 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 262/521] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 24 15:13:43 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:43.730511 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156026b-ea4d-42bc-9be1-9fb538990f2a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:43.796418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094045e-fcc7-4fc1-ad8e-61f07359bc23 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:43.823092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:43.831628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493d849-3822-4f8b-b3ad-01fedbf734a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:43.832237 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493d849-3822-4f8b-b3ad-01fedbf734a1 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:43.832516 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493d849-3822-4f8b-b3ad-01fedbf734a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:43.832813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493d849-3822-4f8b-b3ad-01fedbf734a1 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:43.833559 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493d849-3822-4f8b-b3ad-01fedbf734a1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:43.834081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5156026b-ea4d-42bc-9be1-9fb538990f2a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:43.835026 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493d849-3822-4f8b-b3ad-01fedbf734a1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:43.856866 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 260/522] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:13:43 2022] POST /identity/v3/users => generated 247 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:13:43.857986 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493d849-3822-4f8b-b3ad-01fedbf734a1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:43.880577 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:43.889135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bc40f-513b-4322-a1de-258d7113a58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:43.889774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bc40f-513b-4322-a1de-258d7113a58c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:43.890104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bc40f-513b-4322-a1de-258d7113a58c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:43.890414 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293bc40f-513b-4322-a1de-258d7113a58c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:43.899355 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293bc40f-513b-4322-a1de-258d7113a58c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:43.901141 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293bc40f-513b-4322-a1de-258d7113a58c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:43.916712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-293bc40f-513b-4322-a1de-258d7113a58c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 15:13:43.916712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:43.916712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:43.916712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:43.916712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:43.916712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:43.916712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:43.916712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:43.916712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:43.916712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:43.916712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:43.916712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:43.916712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:13:43.916712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:13:43.916712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:13:43.916712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:43.917704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:43.924182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:43.924182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:43.924182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:13:43.924182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:13:43.924182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:13:43.924182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:13:43.924182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:13:43.924182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:13:43.924182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 15:13:43.924182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:43.924182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 263/523] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:43 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:43.924182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bc40f-513b-4322-a1de-258d7113a58c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:44.006819 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493d849-3822-4f8b-b3ad-01fedbf734a1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:44.048721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a513c-4a5b-4456-9ba7-233c730d6f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:44.049344 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a513c-4a5b-4456-9ba7-233c730d6f9e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:44.049624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a513c-4a5b-4456-9ba7-233c730d6f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:44.049897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a513c-4a5b-4456-9ba7-233c730d6f9e None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:44.050690 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a513c-4a5b-4456-9ba7-233c730d6f9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:44.051000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a513c-4a5b-4456-9ba7-233c730d6f9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:44.053251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a513c-4a5b-4456-9ba7-233c730d6f9e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:44.055422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293bc40f-513b-4322-a1de-258d7113a58c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:44.078254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 261/524] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:13:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:44.078254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a513c-4a5b-4456-9ba7-233c730d6f9e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:44.099791 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28483a-49f7-4132-bb21-b8c34020b8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:44.100570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28483a-49f7-4132-bb21-b8c34020b8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:44.100948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28483a-49f7-4132-bb21-b8c34020b8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:44.101413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f28483a-49f7-4132-bb21-b8c34020b8c4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:44.108436 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28483a-49f7-4132-bb21-b8c34020b8c4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:44.109996 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f28483a-49f7-4132-bb21-b8c34020b8c4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:44.122534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-0f28483a-49f7-4132-bb21-b8c34020b8c4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:44.122534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:44.122534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:44.122534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:44.122534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:44.122534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:44.122534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:44.122534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:44.122534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:44.122534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:44.122534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:44.122534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:44.122534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:13:44.122534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:13:44.122534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:44.123610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:44.124645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:13:44.124645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:13:44.124645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:13:44.124645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:13:44.124645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:13:44.124645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:13:44.124645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:44.124645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:44.130913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 264/525] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:44.130913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28483a-49f7-4132-bb21-b8c34020b8c4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:44.151494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:44.175587 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:44.201419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a513c-4a5b-4456-9ba7-233c730d6f9e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:44.238842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a74af9-46e4-4fb0-b4e9-5745a610e084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:44.239554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a74af9-46e4-4fb0-b4e9-5745a610e084 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:44.240428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a74af9-46e4-4fb0-b4e9-5745a610e084 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:44.240765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a74af9-46e4-4fb0-b4e9-5745a610e084 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:44.254088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f28483a-49f7-4132-bb21-b8c34020b8c4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:44.292989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e3ea4-bdab-4329-8f44-e8eefd6e238e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:44.293597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e3ea4-bdab-4329-8f44-e8eefd6e238e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:44.293872 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e3ea4-bdab-4329-8f44-e8eefd6e238e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:44.294168 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49e3ea4-bdab-4329-8f44-e8eefd6e238e None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:44.294895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e3ea4-bdab-4329-8f44-e8eefd6e238e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:44.295198 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e3ea4-bdab-4329-8f44-e8eefd6e238e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:44.296678 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49e3ea4-bdab-4329-8f44-e8eefd6e238e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:44.316756 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 265/526] 10.222.0.76 () {56 vars in 1139 bytes} [Wed Aug 24 15:13:44 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:44.317601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e3ea4-bdab-4329-8f44-e8eefd6e238e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:44.322150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a74af9-46e4-4fb0-b4e9-5745a610e084 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:44.323820 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a74af9-46e4-4fb0-b4e9-5745a610e084 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:44.417074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:44.419978 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:44.451210 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 262/527] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:44.451886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a74af9-46e4-4fb0-b4e9-5745a610e084 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:44.454751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49e3ea4-bdab-4329-8f44-e8eefd6e238e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:44.505932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d0a6d-f463-40e0-813f-b7d4dd485a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:44.506904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d0a6d-f463-40e0-813f-b7d4dd485a59 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:44.507183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d0a6d-f463-40e0-813f-b7d4dd485a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:44.507463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917d0a6d-f463-40e0-813f-b7d4dd485a59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:44.508195 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d0a6d-f463-40e0-813f-b7d4dd485a59 None admin] RBAC: Adding query filter params (role.id=59a2b50693a646fc8332b835233486ce, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=96c71ea88d9a474c9ff7a30555cfa91f) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:44.508506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d0a6d-f463-40e0-813f-b7d4dd485a59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:44.509948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917d0a6d-f463-40e0-813f-b7d4dd485a59 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:44.515837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 266/528] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:13:44 2022] GET /identity/v3/role_assignments?user.id=96c71ea88d9a474c9ff7a30555cfa91f&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=59a2b50693a646fc8332b835233486ce => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:44.516645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d0a6d-f463-40e0-813f-b7d4dd485a59 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:44.580154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a74af9-46e4-4fb0-b4e9-5745a610e084 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:44.636039 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeeb9a5-9689-48a2-a1d0-4ced6fe4a73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:44.637228 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeeb9a5-9689-48a2-a1d0-4ced6fe4a73f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:44.638801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeeb9a5-9689-48a2-a1d0-4ced6fe4a73f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:44.639401 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeeb9a5-9689-48a2-a1d0-4ced6fe4a73f None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:44.640318 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeeb9a5-9689-48a2-a1d0-4ced6fe4a73f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:44.640806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeeb9a5-9689-48a2-a1d0-4ced6fe4a73f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:44.642500 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeeb9a5-9689-48a2-a1d0-4ced6fe4a73f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:44.660224 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 263/529] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:13:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:44.661510 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeeb9a5-9689-48a2-a1d0-4ced6fe4a73f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:44.680086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917d0a6d-f463-40e0-813f-b7d4dd485a59 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:44.726965 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89ba81-557f-42ee-8ae4-ef2a8296cbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:44.727700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89ba81-557f-42ee-8ae4-ef2a8296cbe8 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:44.728044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89ba81-557f-42ee-8ae4-ef2a8296cbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:44.728429 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c89ba81-557f-42ee-8ae4-ef2a8296cbe8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:44.732403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:44.738585 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:44.789704 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeeb9a5-9689-48a2-a1d0-4ced6fe4a73f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:44.816432 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89ba81-557f-42ee-8ae4-ef2a8296cbe8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:44.818077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c89ba81-557f-42ee-8ae4-ef2a8296cbe8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:44.827555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0eb01d-b89a-41e6-a936-39339084ab69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:44.828914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0eb01d-b89a-41e6-a936-39339084ab69 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:44.829392 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0eb01d-b89a-41e6-a936-39339084ab69 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:44.829836 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0eb01d-b89a-41e6-a936-39339084ab69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:44.841947 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0eb01d-b89a-41e6-a936-39339084ab69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:44.843924 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0eb01d-b89a-41e6-a936-39339084ab69 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:44.860349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8e0eb01d-b89a-41e6-a936-39339084ab69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:44.860349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:44.860349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:44.860349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:44.860349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:44.860349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:44.860349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:44.860349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:44.860349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:44.860349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:44.860349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:44.860349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:44.860349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:44.860349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:44.860349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:44.860349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:44.861291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:44.862120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:44.862120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:44.862120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:44.864435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 264/530] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:44.870641 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0eb01d-b89a-41e6-a936-39339084ab69 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:44.939026 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 267/531] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:44.946607 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89ba81-557f-42ee-8ae4-ef2a8296cbe8 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:44.975064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:44.986160 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0eb01d-b89a-41e6-a936-39339084ab69 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:44.992019 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:45.029740 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4263d-8989-4b33-9f2e-3178d983a606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:45.030769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4263d-8989-4b33-9f2e-3178d983a606 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:45.031247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4263d-8989-4b33-9f2e-3178d983a606 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:45.031748 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac4263d-8989-4b33-9f2e-3178d983a606 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:45.032664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4263d-8989-4b33-9f2e-3178d983a606 None admin] RBAC: Adding query filter params (role.id=59a2b50693a646fc8332b835233486ce, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=8a5fd31a2db944f596912e4f0aa26835) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:45.033148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4263d-8989-4b33-9f2e-3178d983a606 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:45.034777 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac4263d-8989-4b33-9f2e-3178d983a606 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:45.041211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 265/532] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:13:44 2022] GET /identity/v3/role_assignments?user.id=8a5fd31a2db944f596912e4f0aa26835&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=59a2b50693a646fc8332b835233486ce => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:45.042460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4263d-8989-4b33-9f2e-3178d983a606 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:45.113333 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c89ba81-557f-42ee-8ae4-ef2a8296cbe8 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:45.206300 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac4263d-8989-4b33-9f2e-3178d983a606 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:45.233545 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c42eef-74c0-478c-b50e-5d68aa30dfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:45.234687 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c42eef-74c0-478c-b50e-5d68aa30dfab None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:45.236270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c42eef-74c0-478c-b50e-5d68aa30dfab None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:45.237433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c42eef-74c0-478c-b50e-5d68aa30dfab None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:45.238962 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c42eef-74c0-478c-b50e-5d68aa30dfab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:45.240155 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c42eef-74c0-478c-b50e-5d68aa30dfab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:45.243387 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c42eef-74c0-478c-b50e-5d68aa30dfab None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:45.252505 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e203a7b-b22b-4920-966e-187ad186c2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:45.253158 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e203a7b-b22b-4920-966e-187ad186c2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:45.253442 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e203a7b-b22b-4920-966e-187ad186c2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:45.253720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e203a7b-b22b-4920-966e-187ad186c2d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:45.258557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 268/533] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:45.264466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c42eef-74c0-478c-b50e-5d68aa30dfab None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:45.267138 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e203a7b-b22b-4920-966e-187ad186c2d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:45.269615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e203a7b-b22b-4920-966e-187ad186c2d3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:45.286700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7e203a7b-b22b-4920-966e-187ad186c2d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:45.286700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:45.286700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:45.286700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:45.286700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:45.286700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:45.286700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:45.286700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:45.286700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:45.286700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:45.286700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:45.286700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:45.286700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:45.286700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:45.286700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:45.286700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:45.287656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:45.288915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:45.288915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:45.288915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:45.288915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 266/534] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:45.293861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e203a7b-b22b-4920-966e-187ad186c2d3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:45.305398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:45.423795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e203a7b-b22b-4920-966e-187ad186c2d3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:45.465805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc77bc2-6dbd-45ca-b61b-c51fcbc317cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:45.466724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc77bc2-6dbd-45ca-b61b-c51fcbc317cf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:45.467150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc77bc2-6dbd-45ca-b61b-c51fcbc317cf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:45.467567 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc77bc2-6dbd-45ca-b61b-c51fcbc317cf None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:45.468362 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc77bc2-6dbd-45ca-b61b-c51fcbc317cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:45.468834 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc77bc2-6dbd-45ca-b61b-c51fcbc317cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:45.470965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc77bc2-6dbd-45ca-b61b-c51fcbc317cf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:45.487697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 267/535] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:45.494154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc77bc2-6dbd-45ca-b61b-c51fcbc317cf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:45.542603 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:45.568041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c42eef-74c0-478c-b50e-5d68aa30dfab None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:45.582310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:45.626275 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc77bc2-6dbd-45ca-b61b-c51fcbc317cf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:45.668966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e86d0-c98b-47e8-9a66-9e33d6693e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:45.670415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e86d0-c98b-47e8-9a66-9e33d6693e85 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:45.671351 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e86d0-c98b-47e8-9a66-9e33d6693e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:45.671901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1e86d0-c98b-47e8-9a66-9e33d6693e85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:45.680876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e86d0-c98b-47e8-9a66-9e33d6693e85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:45.682718 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1e86d0-c98b-47e8-9a66-9e33d6693e85 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:45.686994 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48d35a-047f-4cdf-976e-52d16b6924bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:45.687956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48d35a-047f-4cdf-976e-52d16b6924bb None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:45.687956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48d35a-047f-4cdf-976e-52d16b6924bb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:45.688167 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48d35a-047f-4cdf-976e-52d16b6924bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:45.700334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2c1e86d0-c98b-47e8-9a66-9e33d6693e85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:45.700334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:45.700334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:45.700334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:45.700334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:45.700334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:45.700334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:45.700334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:45.700334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:45.700334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:45.700334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:45.700334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:45.700334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:13:45.700334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:13:45.700334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:45.701572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:45.702631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:13:45.702631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:13:45.702631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:13:45.702631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:13:45.702631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:13:45.702631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:13:45.702631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:45.702631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:45.704967 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 268/536] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:45.709820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48d35a-047f-4cdf-976e-52d16b6924bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:45.709820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48d35a-047f-4cdf-976e-52d16b6924bb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:45.712137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e86d0-c98b-47e8-9a66-9e33d6693e85 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:45.734130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-1e48d35a-047f-4cdf-976e-52d16b6924bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:45.734130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:45.734130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:45.734130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:45.734130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:45.734130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:45.734130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:45.734130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:45.734130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:45.734130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:45.734130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:45.734130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:45.734130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:13:45.734130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:13:45.734130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:45.735122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:45.736030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:13:45.736030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:13:45.736030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:13:45.736030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:13:45.736030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:13:45.736030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:13:45.736030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:45.736030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:45.736030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 269/537] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 471 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:13:45.749315 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48d35a-047f-4cdf-976e-52d16b6924bb None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:45.942933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1e86d0-c98b-47e8-9a66-9e33d6693e85 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:45.968136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48d35a-047f-4cdf-976e-52d16b6924bb None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:45.985227 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:45.996397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2210a6-f47f-4bbd-a80f-1b4e0c975f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:45.997122 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2210a6-f47f-4bbd-a80f-1b4e0c975f69 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:45.997409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2210a6-f47f-4bbd-a80f-1b4e0c975f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:45.997685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2210a6-f47f-4bbd-a80f-1b4e0c975f69 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:45.998416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2210a6-f47f-4bbd-a80f-1b4e0c975f69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:45.998745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2210a6-f47f-4bbd-a80f-1b4e0c975f69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:46.000373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2210a6-f47f-4bbd-a80f-1b4e0c975f69 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:46.007476 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4116269f-8768-4ff4-9344-7290d97103df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:46.008083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116269f-8768-4ff4-9344-7290d97103df None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:46.008395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116269f-8768-4ff4-9344-7290d97103df None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:46.008734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4116269f-8768-4ff4-9344-7290d97103df None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:46.009334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116269f-8768-4ff4-9344-7290d97103df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:46.009334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116269f-8768-4ff4-9344-7290d97103df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:46.011096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4116269f-8768-4ff4-9344-7290d97103df None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:46.017693 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 269/538] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:13:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:46.019656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2436758-5c2b-4b38-9c08-b4671454a241 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:46.020231 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2436758-5c2b-4b38-9c08-b4671454a241 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:46.021395 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2436758-5c2b-4b38-9c08-b4671454a241 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:46.024275 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 270/539] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:46.025495 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2436758-5c2b-4b38-9c08-b4671454a241 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:46.035466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 270/540] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:13:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:46.037610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70474d44-a58e-4eb9-9645-05fc05bba3a7 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:46.037610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70474d44-a58e-4eb9-9645-05fc05bba3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:46.037884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70474d44-a58e-4eb9-9645-05fc05bba3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:46.059217 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-70474d44-a58e-4eb9-9645-05fc05bba3a7 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:46.060901 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-70474d44-a58e-4eb9-9645-05fc05bba3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:46.078790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:46.149817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2436758-5c2b-4b38-9c08-b4671454a241 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:46.163219 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 271/541] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:46.164690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70474d44-a58e-4eb9-9645-05fc05bba3a7 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:46.214985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197a505-0db0-4b88-8560-0f4a0dd64c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:46.215609 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197a505-0db0-4b88-8560-0f4a0dd64c2d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:46.215890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197a505-0db0-4b88-8560-0f4a0dd64c2d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:46.216163 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5197a505-0db0-4b88-8560-0f4a0dd64c2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:46.216903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197a505-0db0-4b88-8560-0f4a0dd64c2d None admin] RBAC: Adding query filter params (role.id=59a2b50693a646fc8332b835233486ce, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=5427bd0ac88042338b956c10cfd07baa) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:46.217212 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197a505-0db0-4b88-8560-0f4a0dd64c2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:46.218702 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5197a505-0db0-4b88-8560-0f4a0dd64c2d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:46.225046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 271/542] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:13:46 2022] GET /identity/v3/role_assignments?user.id=5427bd0ac88042338b956c10cfd07baa&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=59a2b50693a646fc8332b835233486ce => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:46.297578 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70474d44-a58e-4eb9-9645-05fc05bba3a7 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:46.347677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61bf0db-650a-4f41-a231-2e9fcdc535aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:46.349681 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bf0db-650a-4f41-a231-2e9fcdc535aa None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:46.350324 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bf0db-650a-4f41-a231-2e9fcdc535aa None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:46.350920 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61bf0db-650a-4f41-a231-2e9fcdc535aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:46.352167 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61bf0db-650a-4f41-a231-2e9fcdc535aa None admin] RBAC: Adding query filter params (role.id=59a2b50693a646fc8332b835233486ce, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=194c7492cf954a6a8dd21de66aa24fec) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:46.352881 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61bf0db-650a-4f41-a231-2e9fcdc535aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:46.354857 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61bf0db-650a-4f41-a231-2e9fcdc535aa None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:46.362934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 272/543] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:13:46 2022] GET /identity/v3/role_assignments?user.id=194c7492cf954a6a8dd21de66aa24fec&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=59a2b50693a646fc8332b835233486ce => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:46.487392 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197a505-0db0-4b88-8560-0f4a0dd64c2d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:46.546628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:46.604588 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5197a505-0db0-4b88-8560-0f4a0dd64c2d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:46.640312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb552d4b-1d71-4b02-b63a-61957edbabe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:46.640955 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb552d4b-1d71-4b02-b63a-61957edbabe2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:46.641232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb552d4b-1d71-4b02-b63a-61957edbabe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:46.641502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb552d4b-1d71-4b02-b63a-61957edbabe2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:46.653610 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb552d4b-1d71-4b02-b63a-61957edbabe2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:46.658853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-fb552d4b-1d71-4b02-b63a-61957edbabe2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:46.658853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:46.658853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:46.658853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:46.658853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:46.658853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:46.658853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:46.658853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:46.658853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:46.658853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:46.658853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:46.658853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:46.658853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:46.658853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:46.658853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:46.658853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:46.659917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:46.659917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:46.659917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:46.659917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:46.659917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:46.659917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:46.659917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:46.659917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:46.659917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:46.659917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:46.659917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:46.659917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:46.659917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:46.659917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:46.659917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:46.659917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:46.660688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:46.660688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:46.660688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:46.660688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:46.660688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:46.660688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:46.660688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:46.660688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:46.660688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:46.660688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:46.660688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:46.660688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:46.660688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 272/544] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:46.666950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61bf0db-650a-4f41-a231-2e9fcdc535aa None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:46.686042 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:46.823192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61bf0db-650a-4f41-a231-2e9fcdc535aa None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:46.824786 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873249d-efe9-4daa-81af-c62f0e5a6c6f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:46.825074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873249d-efe9-4daa-81af-c62f0e5a6c6f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:46.825357 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7873249d-efe9-4daa-81af-c62f0e5a6c6f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:46.826986 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 273/545] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:46.835608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55700db-aff9-4361-b4f0-81d43a355cc8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:46.835893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55700db-aff9-4361-b4f0-81d43a355cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:46.836169 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55700db-aff9-4361-b4f0-81d43a355cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:46.863645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1048186-69bc-4557-bd0f-4664dea45228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:46.864255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1048186-69bc-4557-bd0f-4664dea45228 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:46.864562 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1048186-69bc-4557-bd0f-4664dea45228 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:46.865413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1048186-69bc-4557-bd0f-4664dea45228 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:46.866088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1048186-69bc-4557-bd0f-4664dea45228 None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:46.870607 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b55700db-aff9-4361-b4f0-81d43a355cc8 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:46.871228 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1048186-69bc-4557-bd0f-4664dea45228 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:46.872652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1048186-69bc-4557-bd0f-4664dea45228 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:46.874938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b55700db-aff9-4361-b4f0-81d43a355cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:46.892567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 273/546] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:13:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:46.896823 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1048186-69bc-4557-bd0f-4664dea45228 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:46.996306 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 274/547] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:47.022113 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:47.065166 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1048186-69bc-4557-bd0f-4664dea45228 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:47.106993 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5930a8-2631-492b-aad1-91883fe39075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:47.107622 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5930a8-2631-492b-aad1-91883fe39075 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:47.107897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5930a8-2631-492b-aad1-91883fe39075 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:47.108164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5930a8-2631-492b-aad1-91883fe39075 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:47.120250 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5930a8-2631-492b-aad1-91883fe39075 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:47.121657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5930a8-2631-492b-aad1-91883fe39075 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:47.137389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-9a5930a8-2631-492b-aad1-91883fe39075 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:47.137389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:47.137389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:47.137389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:47.137389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:47.137389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:47.137389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:47.137389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:47.137389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:47.137389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:47.137389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:47.137389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:47.137389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:47.137389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:47.137389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:47.137389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:47.138826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:47.139970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:47.139970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:47.139970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:47.139970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 274/548] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:47.144581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55700db-aff9-4361-b4f0-81d43a355cc8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:47.235443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:47.335149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55700db-aff9-4361-b4f0-81d43a355cc8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:47.382810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ed0f0-0a1f-45dd-bf70-866c044487e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:47.383686 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ed0f0-0a1f-45dd-bf70-866c044487e6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:47.384124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ed0f0-0a1f-45dd-bf70-866c044487e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:47.384575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ed0f0-0a1f-45dd-bf70-866c044487e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:47.385575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ed0f0-0a1f-45dd-bf70-866c044487e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:47.386085 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ed0f0-0a1f-45dd-bf70-866c044487e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:47.387929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ed0f0-0a1f-45dd-bf70-866c044487e6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:47.405428 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62f1d2-141a-4c93-9e9a-f681689cf1f1 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:47.405850 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62f1d2-141a-4c93-9e9a-f681689cf1f1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:47.406235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62f1d2-141a-4c93-9e9a-f681689cf1f1 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:47.407980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 275/550] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:47.408095 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 275/550] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:47.413096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62f1d2-141a-4c93-9e9a-f681689cf1f1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:47.415699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd3082-b3f5-4d0e-8f3c-13c0a39d74eb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:47.416163 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd3082-b3f5-4d0e-8f3c-13c0a39d74eb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:47.416564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd3082-b3f5-4d0e-8f3c-13c0a39d74eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:47.447562 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-66dd3082-b3f5-4d0e-8f3c-13c0a39d74eb None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:47.450630 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-66dd3082-b3f5-4d0e-8f3c-13c0a39d74eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:47.544824 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 276/551] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:13:47.545845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd3082-b3f5-4d0e-8f3c-13c0a39d74eb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:47.579522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:47.602502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62f1d2-141a-4c93-9e9a-f681689cf1f1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:47.630645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:47.644632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4dc92-13be-4c4a-be20-fe2c10e75df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:47.645495 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4dc92-13be-4c4a-be20-fe2c10e75df9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:47.646350 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4dc92-13be-4c4a-be20-fe2c10e75df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:47.646760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f4dc92-13be-4c4a-be20-fe2c10e75df9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:47.656485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4dc92-13be-4c4a-be20-fe2c10e75df9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:47.658550 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f4dc92-13be-4c4a-be20-fe2c10e75df9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:47.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-61f4dc92-13be-4c4a-be20-fe2c10e75df9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:13:47.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:47.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:47.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:47.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:47.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:47.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:47.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:47.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:47.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:47.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:47.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:47.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:13:47.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:13:47.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:13:47.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:47.675143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:47.676260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:47.676260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:47.676260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:13:47.676260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:13:47.676260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:13:47.676260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:13:47.676260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:13:47.676260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:13:47.676260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:13:47.676260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:47.679010 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 276/552] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:13:47 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:47.685334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4dc92-13be-4c4a-be20-fe2c10e75df9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:47.738647 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd3082-b3f5-4d0e-8f3c-13c0a39d74eb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:47.824811 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f4dc92-13be-4c4a-be20-fe2c10e75df9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:47.827097 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8169cec-3194-42f6-b3ae-7101867dbd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:47.828296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8169cec-3194-42f6-b3ae-7101867dbd5d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:47.828296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8169cec-3194-42f6-b3ae-7101867dbd5d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:47.828517 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8169cec-3194-42f6-b3ae-7101867dbd5d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:47.847018 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8169cec-3194-42f6-b3ae-7101867dbd5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:47.854657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e8169cec-3194-42f6-b3ae-7101867dbd5d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:47.854657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:47.854657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:47.854657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:47.854657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:47.854657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:47.854657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:47.854657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:47.854657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:47.854657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:47.854657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:47.854657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:47.854657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:47.854657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:47.854657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:47.854657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:47.855808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:47.855808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:47.855808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:47.855808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:47.855808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:47.855808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:47.855808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:47.855808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:47.855808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:47.855808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:47.855808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:47.855808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:47.855808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:47.855808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:47.855808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:47.855808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:47.856954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:47.856954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:47.856954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:47.856954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:47.856954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:47.856954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:47.856954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:47.856954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:47.856954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:47.856954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:47.856954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:47.856954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:47.856954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 277/553] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:47.857712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8169cec-3194-42f6-b3ae-7101867dbd5d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:47.873259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facccd5b-815e-46b5-b82f-357c19a2b2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:47.873797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facccd5b-815e-46b5-b82f-357c19a2b2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:47.874089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facccd5b-815e-46b5-b82f-357c19a2b2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:47.874454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facccd5b-815e-46b5-b82f-357c19a2b2a7 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:47.875188 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facccd5b-815e-46b5-b82f-357c19a2b2a7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:47.875535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facccd5b-815e-46b5-b82f-357c19a2b2a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:47.876968 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facccd5b-815e-46b5-b82f-357c19a2b2a7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:47.897515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 277/554] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 24 15:13:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:47.902249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facccd5b-815e-46b5-b82f-357c19a2b2a7 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:47.981041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:47.985258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:47.998805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8169cec-3194-42f6-b3ae-7101867dbd5d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:48.039866 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a6487-b3dd-4a1b-b685-d674ca8a300a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:48.040897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a6487-b3dd-4a1b-b685-d674ca8a300a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:48.041385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a6487-b3dd-4a1b-b685-d674ca8a300a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:48.041828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a6487-b3dd-4a1b-b685-d674ca8a300a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:48.055829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a6487-b3dd-4a1b-b685-d674ca8a300a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:48.061286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-cf0a6487-b3dd-4a1b-b685-d674ca8a300a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:48.061286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:48.061286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:48.061286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:48.061286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:48.061286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:48.061286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:48.061286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:48.061286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:48.061286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:48.061286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:48.061286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:48.061286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:48.061286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:48.061286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:48.061286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:48.062316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:48.062316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:48.062316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:48.062316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:48.062316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:48.062316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:48.062316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:48.062316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:48.062316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:48.062316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:48.062316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:48.062316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:48.062316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:48.062316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:48.062316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:48.062316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:48.063124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:48.063124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:48.063124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:48.063124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:48.063124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:48.063124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:48.063124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:48.063124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:48.063124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:48.063124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:48.063124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:48.063124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:48.063763 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facccd5b-815e-46b5-b82f-357c19a2b2a7 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:48.066501 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 278/555] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:48.067547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a6487-b3dd-4a1b-b685-d674ca8a300a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:48.110043 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc05a11-7509-4446-8ead-db26baef297e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:48.110804 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc05a11-7509-4446-8ead-db26baef297e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:48.111163 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc05a11-7509-4446-8ead-db26baef297e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:48.111502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc05a11-7509-4446-8ead-db26baef297e None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:48.112245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc05a11-7509-4446-8ead-db26baef297e None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:48.112609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc05a11-7509-4446-8ead-db26baef297e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:48.114197 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc05a11-7509-4446-8ead-db26baef297e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:48.126963 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 278/556] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:13:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:48.127879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc05a11-7509-4446-8ead-db26baef297e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:48.188135 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a6487-b3dd-4a1b-b685-d674ca8a300a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:48.230461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677355df-75da-48fe-92e9-073c00ba2123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:48.231138 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677355df-75da-48fe-92e9-073c00ba2123 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:48.231524 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677355df-75da-48fe-92e9-073c00ba2123 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:48.231800 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677355df-75da-48fe-92e9-073c00ba2123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:48.256084 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc05a11-7509-4446-8ead-db26baef297e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:48.257652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:48.259849 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:48.294474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be492529-af4d-4879-a1b1-38c4d434b6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:48.295194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be492529-af4d-4879-a1b1-38c4d434b6a8 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:48.295473 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be492529-af4d-4879-a1b1-38c4d434b6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:48.296538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be492529-af4d-4879-a1b1-38c4d434b6a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:48.297755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be492529-af4d-4879-a1b1-38c4d434b6a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:48.298123 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be492529-af4d-4879-a1b1-38c4d434b6a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:48.299897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be492529-af4d-4879-a1b1-38c4d434b6a8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:48.317978 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 279/557] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:13:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:48.317978 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be492529-af4d-4879-a1b1-38c4d434b6a8 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:48.336552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677355df-75da-48fe-92e9-073c00ba2123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:48.338483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677355df-75da-48fe-92e9-073c00ba2123 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:48.455731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be492529-af4d-4879-a1b1-38c4d434b6a8 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:48.482441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 279/558] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:48.483736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677355df-75da-48fe-92e9-073c00ba2123 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:48.509442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358127d-ed4f-45b1-9ea4-ab914f461a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:48.511089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358127d-ed4f-45b1-9ea4-ab914f461a30 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:48.511830 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358127d-ed4f-45b1-9ea4-ab914f461a30 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:48.512407 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e358127d-ed4f-45b1-9ea4-ab914f461a30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:48.532655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358127d-ed4f-45b1-9ea4-ab914f461a30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:48.535025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e358127d-ed4f-45b1-9ea4-ab914f461a30 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:48.553374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-e358127d-ed4f-45b1-9ea4-ab914f461a30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:48.553374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:48.553374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:48.553374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:48.553374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:48.553374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:48.553374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:48.553374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:48.553374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:48.553374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:48.553374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:48.553374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:48.553374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:48.553374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:48.553374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:48.553374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:48.554435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:48.555314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:48.555314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:48.555314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:48.558440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 280/559] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:48.559344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358127d-ed4f-45b1-9ea4-ab914f461a30 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:48.562318 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:48.587825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:48.648843 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677355df-75da-48fe-92e9-073c00ba2123 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:48.690834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e358127d-ed4f-45b1-9ea4-ab914f461a30 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:48.704101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8f7c7-118d-4351-abfe-5b1800c46ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:48.704822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8f7c7-118d-4351-abfe-5b1800c46ccc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:48.705277 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8f7c7-118d-4351-abfe-5b1800c46ccc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:48.705677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e8f7c7-118d-4351-abfe-5b1800c46ccc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:48.720094 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8f7c7-118d-4351-abfe-5b1800c46ccc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:48.721731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e8f7c7-118d-4351-abfe-5b1800c46ccc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:48.743273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb219762-7343-4a34-b2ce-eefdd673d010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:48.743734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d2e8f7c7-118d-4351-abfe-5b1800c46ccc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:48.743734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:48.743734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:48.743734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:48.743734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:48.743734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:48.743734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:48.743734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:48.743734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:48.743734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:48.743734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:48.743734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:48.743734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:48.743734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:48.743734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:48.743734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:48.745443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:48.746767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:48.746767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:48.746767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:48.747064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219762-7343-4a34-b2ce-eefdd673d010 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:48.747064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219762-7343-4a34-b2ce-eefdd673d010 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:48.747064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb219762-7343-4a34-b2ce-eefdd673d010 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:48.751229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 280/560] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:48.752215 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8f7c7-118d-4351-abfe-5b1800c46ccc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:48.762054 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb219762-7343-4a34-b2ce-eefdd673d010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:48.764715 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb219762-7343-4a34-b2ce-eefdd673d010 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:48.782104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-fb219762-7343-4a34-b2ce-eefdd673d010 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:48.782104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:48.782104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:48.782104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:48.782104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:48.782104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:48.782104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:48.782104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:48.782104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:48.782104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:48.782104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:48.782104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:48.782104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:48.782104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:48.782104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:48.782104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:48.783138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:48.784045 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:48.784045 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:48.784045 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:48.784045 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 281/561] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:48.785777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d1621-84e3-4d1f-a311-7ef3bcb5b182 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:48.785777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d1621-84e3-4d1f-a311-7ef3bcb5b182 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:48.785777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1d1621-84e3-4d1f-a311-7ef3bcb5b182 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:48.790273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 282/562] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:48.790273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d1621-84e3-4d1f-a311-7ef3bcb5b182 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:48.870924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:48.900638 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e8f7c7-118d-4351-abfe-5b1800c46ccc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:48.902555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:48.915885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1d1621-84e3-4d1f-a311-7ef3bcb5b182 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:48.950352 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4a93e-2d86-45cb-b2ef-a5ea702f780f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:48.951018 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4a93e-2d86-45cb-b2ef-a5ea702f780f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:48.951307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4a93e-2d86-45cb-b2ef-a5ea702f780f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:48.951479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d4a93e-2d86-45cb-b2ef-a5ea702f780f None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:48.952324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4a93e-2d86-45cb-b2ef-a5ea702f780f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:48.952618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4a93e-2d86-45cb-b2ef-a5ea702f780f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:48.953981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d4a93e-2d86-45cb-b2ef-a5ea702f780f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:48.957238 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec538b-5c5d-4129-a073-f3f75b8d0f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:48.957849 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec538b-5c5d-4129-a073-f3f75b8d0f93 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:48.958304 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec538b-5c5d-4129-a073-f3f75b8d0f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:48.958584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec538b-5c5d-4129-a073-f3f75b8d0f93 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:48.959232 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec538b-5c5d-4129-a073-f3f75b8d0f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:48.959530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec538b-5c5d-4129-a073-f3f75b8d0f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:48.960915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec538b-5c5d-4129-a073-f3f75b8d0f93 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:48.968125 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 281/563] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:48.968969 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4a93e-2d86-45cb-b2ef-a5ea702f780f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:48.977179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 283/564] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:48.978730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef18bb8-9e3e-478b-ae0d-977d56a384cd None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:48.979014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef18bb8-9e3e-478b-ae0d-977d56a384cd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:48.979283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef18bb8-9e3e-478b-ae0d-977d56a384cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:49.000507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-aef18bb8-9e3e-478b-ae0d-977d56a384cd None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:49.001709 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-aef18bb8-9e3e-478b-ae0d-977d56a384cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:49.087897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d4a93e-2d86-45cb-b2ef-a5ea702f780f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:49.088449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 284/565] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:49.104840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f82d6-c707-4c0f-a5bb-2633f34a6ae8 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:49.105772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f82d6-c707-4c0f-a5bb-2633f34a6ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:49.105772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269f82d6-c707-4c0f-a5bb-2633f34a6ae8 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:49.106808 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 285/566] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:49 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:49.107574 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f82d6-c707-4c0f-a5bb-2633f34a6ae8 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:49.140110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a551f-beec-4369-951e-c153b9e3f21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:49.140877 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a551f-beec-4369-951e-c153b9e3f21a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:49.141265 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a551f-beec-4369-951e-c153b9e3f21a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:49.141466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a551f-beec-4369-951e-c153b9e3f21a None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:49.142180 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a551f-beec-4369-951e-c153b9e3f21a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:49.142512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a551f-beec-4369-951e-c153b9e3f21a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:49.144017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a551f-beec-4369-951e-c153b9e3f21a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:49.165231 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 282/567] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:49.165231 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a551f-beec-4369-951e-c153b9e3f21a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:49.188126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:49.193079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:49.244989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269f82d6-c707-4c0f-a5bb-2633f34a6ae8 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:49.291277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2047dcc-5201-4ab1-92c2-175b6265a781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:49.291968 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2047dcc-5201-4ab1-92c2-175b6265a781 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:49.292243 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2047dcc-5201-4ab1-92c2-175b6265a781 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:49.292499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2047dcc-5201-4ab1-92c2-175b6265a781 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:49.303751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2047dcc-5201-4ab1-92c2-175b6265a781 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:49.305525 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2047dcc-5201-4ab1-92c2-175b6265a781 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:49.310103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a551f-beec-4369-951e-c153b9e3f21a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:49.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-b2047dcc-5201-4ab1-92c2-175b6265a781 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 15:13:49.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:49.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:49.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:49.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:49.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:49.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:49.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:49.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:49.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:49.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:49.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:49.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:13:49.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:13:49.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:13:49.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:49.321621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:49.322749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:49.322749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:49.322749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:13:49.322749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:13:49.322749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:13:49.322749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:13:49.322749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:13:49.322749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:13:49.322749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 15:13:49.322749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:49.322749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 286/568] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:49.323892 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6bd51-bfac-4706-9339-2f3e60690246 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:49.324197 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6bd51-bfac-4706-9339-2f3e60690246 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:49.324505 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd6bd51-bfac-4706-9339-2f3e60690246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:49.347731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2dd6bd51-bfac-4706-9339-2f3e60690246 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:49.349685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2dd6bd51-bfac-4706-9339-2f3e60690246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:49.367001 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd94ca-cf29-4d97-a410-64e250d0a029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:49.367783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd94ca-cf29-4d97-a410-64e250d0a029 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:49.368147 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd94ca-cf29-4d97-a410-64e250d0a029 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:49.368468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dd94ca-cf29-4d97-a410-64e250d0a029 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:49.377458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd94ca-cf29-4d97-a410-64e250d0a029 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:49.379207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dd94ca-cf29-4d97-a410-64e250d0a029 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:49.391280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a1dd94ca-cf29-4d97-a410-64e250d0a029 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:13:49.391280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:49.391280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:49.391280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:49.391280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:49.391280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:49.391280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:49.391280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:49.391280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:49.391280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:49.391280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:49.391280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:49.391280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:13:49.391280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:13:49.391280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:13:49.391280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:49.393054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:49.395049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:49.395049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:49.395049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:13:49.395049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:13:49.395049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:13:49.395049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:13:49.395049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:13:49.395049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:13:49.395049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:13:49.395049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:49.395049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 283/569] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 15:13:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:49.395049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd94ca-cf29-4d97-a410-64e250d0a029 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:49.437465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 287/570] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:49.438466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6bd51-bfac-4706-9339-2f3e60690246 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:49.507209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:49.515271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:49.535029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dd94ca-cf29-4d97-a410-64e250d0a029 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:49.577136 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2af69d-8a5f-455e-80bb-b1c4c9bab17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:49.577800 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2af69d-8a5f-455e-80bb-b1c4c9bab17c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:49.578083 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2af69d-8a5f-455e-80bb-b1c4c9bab17c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:49.578392 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2af69d-8a5f-455e-80bb-b1c4c9bab17c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:49.583968 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd6bd51-bfac-4706-9339-2f3e60690246 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:49.586091 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2af69d-8a5f-455e-80bb-b1c4c9bab17c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:49.588434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2af69d-8a5f-455e-80bb-b1c4c9bab17c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:49.603601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-cc2af69d-8a5f-455e-80bb-b1c4c9bab17c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:49.603601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:49.603601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:49.603601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:49.603601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:49.603601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:49.603601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:49.603601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:49.603601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:49.603601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:49.603601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:49.603601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:49.603601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:13:49.603601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:13:49.603601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:49.604504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:49.605398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:13:49.605398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:13:49.605398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:13:49.605398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:13:49.605398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:13:49.605398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:13:49.605398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:49.605398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:49.609401 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 284/571] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:49.610234 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2af69d-8a5f-455e-80bb-b1c4c9bab17c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:49.622362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56a361-db54-483b-b5d8-66274385a42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:49.622972 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56a361-db54-483b-b5d8-66274385a42f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:49.623238 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56a361-db54-483b-b5d8-66274385a42f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:49.623517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b56a361-db54-483b-b5d8-66274385a42f None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:49.624250 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56a361-db54-483b-b5d8-66274385a42f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:49.624556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56a361-db54-483b-b5d8-66274385a42f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:49.625998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b56a361-db54-483b-b5d8-66274385a42f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:49.643970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 288/572] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 24 15:13:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:49.645362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56a361-db54-483b-b5d8-66274385a42f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:49.728999 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2af69d-8a5f-455e-80bb-b1c4c9bab17c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:49.751966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b56a361-db54-483b-b5d8-66274385a42f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:49.760227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:49.767784 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf9efe-0bdd-4e3c-876f-dc0964474eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:49.768424 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf9efe-0bdd-4e3c-876f-dc0964474eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:49.768715 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf9efe-0bdd-4e3c-876f-dc0964474eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:49.768981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cf9efe-0bdd-4e3c-876f-dc0964474eb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:49.779996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cf9efe-0bdd-4e3c-876f-dc0964474eb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:49.785209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-58cf9efe-0bdd-4e3c-876f-dc0964474eb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:49.785209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:49.785209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:49.785209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:49.785209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:49.785209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:49.785209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:49.785209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:49.785209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:49.785209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:49.785209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:49.785209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:49.785209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:49.785209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:49.785209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:49.785209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:49.786133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:49.786133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:49.786133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:49.786133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:49.786133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:49.786133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:49.786133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:49.786133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:49.786133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:49.786133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:49.786133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:49.786133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:49.786133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:49.786133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:49.786133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:49.786133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:49.787103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:49.787103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:49.787103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:49.787103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:49.787103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:49.787103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:49.787103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:49.787103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:49.787103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:49.787103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:49.787103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:49.787103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:49.787103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 285/573] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:49.787791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf9efe-0bdd-4e3c-876f-dc0964474eb2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:49.790101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:49.793673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba50ac-5481-4a7d-9fb2-8abc20b3694f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:49.794629 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba50ac-5481-4a7d-9fb2-8abc20b3694f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:49.794963 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba50ac-5481-4a7d-9fb2-8abc20b3694f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:49.795232 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba50ac-5481-4a7d-9fb2-8abc20b3694f None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:49.795976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba50ac-5481-4a7d-9fb2-8abc20b3694f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:49.796279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba50ac-5481-4a7d-9fb2-8abc20b3694f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:49.797723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba50ac-5481-4a7d-9fb2-8abc20b3694f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:49.816246 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 289/574] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 15:13:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:49.817471 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba50ac-5481-4a7d-9fb2-8abc20b3694f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:49.897639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cf9efe-0bdd-4e3c-876f-dc0964474eb2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:49.926040 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba50ac-5481-4a7d-9fb2-8abc20b3694f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:49.934545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757346da-719e-4237-8cbf-10fe0851c398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:49.935149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757346da-719e-4237-8cbf-10fe0851c398 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:49.935423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757346da-719e-4237-8cbf-10fe0851c398 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:49.935687 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757346da-719e-4237-8cbf-10fe0851c398 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:49.936375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757346da-719e-4237-8cbf-10fe0851c398 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:49.936674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757346da-719e-4237-8cbf-10fe0851c398 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:49.938091 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757346da-719e-4237-8cbf-10fe0851c398 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:49.950134 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 286/575] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:13:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:49.950798 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757346da-719e-4237-8cbf-10fe0851c398 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:49.962812 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb7a7a-f83a-441e-8f1a-f40a9ea483c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:49.963436 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb7a7a-f83a-441e-8f1a-f40a9ea483c4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:49.963710 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb7a7a-f83a-441e-8f1a-f40a9ea483c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:49.963977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb7a7a-f83a-441e-8f1a-f40a9ea483c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:49.974809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb7a7a-f83a-441e-8f1a-f40a9ea483c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:49.976201 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb7a7a-f83a-441e-8f1a-f40a9ea483c4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:49.994552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-26cb7a7a-f83a-441e-8f1a-f40a9ea483c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:49.994552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:49.994552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:49.994552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:49.994552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:49.994552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:49.994552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:49.994552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:49.994552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:49.994552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:49.994552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:49.994552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:49.994552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:49.994552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:49.994552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:49.994552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:49.995716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:49.996909 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:49.996909 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:49.996909 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:49.998539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 290/576] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:49.999203 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb7a7a-f83a-441e-8f1a-f40a9ea483c4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:50.014646 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:50.020735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:50.055351 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757346da-719e-4237-8cbf-10fe0851c398 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:50.089795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951b065-ab32-403e-94bb-25d97a2f08bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:50.090515 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951b065-ab32-403e-94bb-25d97a2f08bb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:50.090720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951b065-ab32-403e-94bb-25d97a2f08bb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:50.090992 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951b065-ab32-403e-94bb-25d97a2f08bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:50.101501 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c951b065-ab32-403e-94bb-25d97a2f08bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:50.103845 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb7a7a-f83a-441e-8f1a-f40a9ea483c4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:50.106242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c951b065-ab32-403e-94bb-25d97a2f08bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:50.106242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:50.106242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:50.106242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:50.106242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:50.106242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:50.106242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:50.106242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:50.106242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:50.106242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:50.106242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:50.106242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:50.106242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:50.106242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:50.106242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:50.106242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:50.107236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:50.107236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:50.107236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:50.107236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:50.107236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:50.107236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:50.107236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:50.107236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:50.107236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:50.107236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:50.107236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:50.107236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:50.107236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:50.107236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:50.107236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:50.107236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:50.108048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:50.108048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:50.108048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:50.108048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:50.108048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:50.108048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:50.108048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:50.108048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:50.108048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:50.108048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:50.108048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:50.108048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:50.108048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 287/577] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:50.108742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951b065-ab32-403e-94bb-25d97a2f08bb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:50.140156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f76d0-9e7a-4dcc-9eee-d38cc3d82a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:50.140765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f76d0-9e7a-4dcc-9eee-d38cc3d82a9c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:50.141041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f76d0-9e7a-4dcc-9eee-d38cc3d82a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:50.141313 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f76d0-9e7a-4dcc-9eee-d38cc3d82a9c None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:50.141971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f76d0-9e7a-4dcc-9eee-d38cc3d82a9c None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:50.142299 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f76d0-9e7a-4dcc-9eee-d38cc3d82a9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:50.143692 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f76d0-9e7a-4dcc-9eee-d38cc3d82a9c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:50.152380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 291/578] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:13:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:50.153297 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f76d0-9e7a-4dcc-9eee-d38cc3d82a9c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:50.218639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c951b065-ab32-403e-94bb-25d97a2f08bb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:50.237481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:50.249161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:50.261521 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdca026-9392-41ef-a72e-3f088c3fdb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:50.262150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdca026-9392-41ef-a72e-3f088c3fdb2c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:50.262445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdca026-9392-41ef-a72e-3f088c3fdb2c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:50.262710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdca026-9392-41ef-a72e-3f088c3fdb2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:50.264769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f76d0-9e7a-4dcc-9eee-d38cc3d82a9c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:50.273570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdca026-9392-41ef-a72e-3f088c3fdb2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:50.275060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdca026-9392-41ef-a72e-3f088c3fdb2c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:50.291496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-3bdca026-9392-41ef-a72e-3f088c3fdb2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:50.291496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:50.291496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:50.291496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:50.291496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:50.291496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:50.291496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:50.291496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:50.291496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:50.291496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:50.291496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:50.291496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:50.291496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:50.291496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:50.291496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:50.291496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:50.292644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:50.293698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:50.293698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:50.293698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:50.293698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 288/579] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:50.293698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdca026-9392-41ef-a72e-3f088c3fdb2c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:50.304014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c257c-0fef-40ff-ac01-e398037f5b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:50.304617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c257c-0fef-40ff-ac01-e398037f5b6f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:50.304890 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c257c-0fef-40ff-ac01-e398037f5b6f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:50.305154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c257c-0fef-40ff-ac01-e398037f5b6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:50.305874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c257c-0fef-40ff-ac01-e398037f5b6f None admin] RBAC: Adding query filter params (role.id=59a2b50693a646fc8332b835233486ce, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=0679072b37db41e880a95b738bea47ea) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:50.306176 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c257c-0fef-40ff-ac01-e398037f5b6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:50.307633 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c257c-0fef-40ff-ac01-e398037f5b6f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:50.313524 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 292/580] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:13:50 2022] GET /identity/v3/role_assignments?user.id=0679072b37db41e880a95b738bea47ea&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=59a2b50693a646fc8332b835233486ce => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:50.314900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c257c-0fef-40ff-ac01-e398037f5b6f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:50.414295 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdca026-9392-41ef-a72e-3f088c3fdb2c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:50.456618 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c257c-0fef-40ff-ac01-e398037f5b6f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:50.459378 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c634e60-ccdd-4e11-a6bc-11efdbfa8216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:50.460086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c634e60-ccdd-4e11-a6bc-11efdbfa8216 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:50.460441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c634e60-ccdd-4e11-a6bc-11efdbfa8216 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:50.460776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c634e60-ccdd-4e11-a6bc-11efdbfa8216 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:50.461519 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c634e60-ccdd-4e11-a6bc-11efdbfa8216 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:50.461974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c634e60-ccdd-4e11-a6bc-11efdbfa8216 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:50.463565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c634e60-ccdd-4e11-a6bc-11efdbfa8216 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:50.476982 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 289/581] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:50.477839 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c634e60-ccdd-4e11-a6bc-11efdbfa8216 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:50.512574 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229804a-1a44-4828-8ad4-c06697d8024a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:50.513855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229804a-1a44-4828-8ad4-c06697d8024a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:50.514310 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229804a-1a44-4828-8ad4-c06697d8024a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:50.514721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229804a-1a44-4828-8ad4-c06697d8024a None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:50.516571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a229804a-1a44-4828-8ad4-c06697d8024a None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:50.516799 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a229804a-1a44-4828-8ad4-c06697d8024a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:50.518269 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a229804a-1a44-4828-8ad4-c06697d8024a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:50.519936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:50.532844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 293/582] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:13:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:50.533794 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229804a-1a44-4828-8ad4-c06697d8024a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:50.538529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:50.612754 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c634e60-ccdd-4e11-a6bc-11efdbfa8216 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:50.656862 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229804a-1a44-4828-8ad4-c06697d8024a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:50.657809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32f6f9-e45d-4cb6-aa50-6fd861b5fbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:50.659063 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32f6f9-e45d-4cb6-aa50-6fd861b5fbf8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:50.659677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32f6f9-e45d-4cb6-aa50-6fd861b5fbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:50.660127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f32f6f9-e45d-4cb6-aa50-6fd861b5fbf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:50.680652 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32f6f9-e45d-4cb6-aa50-6fd861b5fbf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:50.683175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f32f6f9-e45d-4cb6-aa50-6fd861b5fbf8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:50.695352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c3939-91ff-4d75-9829-56f586183f28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:50.696049 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c3939-91ff-4d75-9829-56f586183f28 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:50.696394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c3939-91ff-4d75-9829-56f586183f28 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:50.697178 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7c3939-91ff-4d75-9829-56f586183f28 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:50.697896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c3939-91ff-4d75-9829-56f586183f28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:50.698326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c3939-91ff-4d75-9829-56f586183f28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:50.700153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7c3939-91ff-4d75-9829-56f586183f28 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:50.709029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0f32f6f9-e45d-4cb6-aa50-6fd861b5fbf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:50.709029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:50.709029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:50.709029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:50.709029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:50.709029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:50.709029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:50.709029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:50.709029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:50.709029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:50.709029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:50.709029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:50.709029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:50.709029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:50.709029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:50.709029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:50.710182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:50.711252 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:50.711252 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:50.711252 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:50.711430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 290/583] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:50.712619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32f6f9-e45d-4cb6-aa50-6fd861b5fbf8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:50.717709 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 294/584] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:50.718817 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c3939-91ff-4d75-9829-56f586183f28 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:50.802726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:50.834650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:50.851879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7c3939-91ff-4d75-9829-56f586183f28 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:50.857436 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f32f6f9-e45d-4cb6-aa50-6fd861b5fbf8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:50.890667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba842165-e4a1-4199-a372-b32930ef1d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:50.891607 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba842165-e4a1-4199-a372-b32930ef1d36 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:50.891983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba842165-e4a1-4199-a372-b32930ef1d36 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:50.892356 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba842165-e4a1-4199-a372-b32930ef1d36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:50.898646 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923a1a3-86a4-45b1-9ff1-b038e8723a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:50.899284 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923a1a3-86a4-45b1-9ff1-b038e8723a60 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:50.899690 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923a1a3-86a4-45b1-9ff1-b038e8723a60 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:50.899965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2923a1a3-86a4-45b1-9ff1-b038e8723a60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:50.903820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba842165-e4a1-4199-a372-b32930ef1d36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:50.905419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba842165-e4a1-4199-a372-b32930ef1d36 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:50.907978 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2923a1a3-86a4-45b1-9ff1-b038e8723a60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:50.909564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2923a1a3-86a4-45b1-9ff1-b038e8723a60 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:50.922079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ba842165-e4a1-4199-a372-b32930ef1d36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:50.922079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:50.922079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:50.922079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:50.922079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:50.922079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:50.922079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:50.922079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:50.922079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:50.922079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:50.922079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:50.922079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:50.922079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:50.922079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:50.922079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:50.922079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:50.923274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:50.924402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:50.924402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:50.924402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:50.924611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2923a1a3-86a4-45b1-9ff1-b038e8723a60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:50.924611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:50.924611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:50.924611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:50.924611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:50.924611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:50.924611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:50.924611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:50.924611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:50.924611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:50.924611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:50.924611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:50.924611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:13:50.924611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:13:50.924611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:50.925613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:50.926743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:13:50.926743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:13:50.926743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:13:50.926743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:13:50.926743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:13:50.926743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:13:50.926743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:50.926743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:50.933105 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 295/585] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:50.933257 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 291/586] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:50.933412 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba842165-e4a1-4199-a372-b32930ef1d36 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:50.933509 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923a1a3-86a4-45b1-9ff1-b038e8723a60 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:51.043794 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2923a1a3-86a4-45b1-9ff1-b038e8723a60 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:51.049704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba842165-e4a1-4199-a372-b32930ef1d36 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:51.068440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:51.081025 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235d072-3ba3-4854-b2ab-5906be021b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:51.081658 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235d072-3ba3-4854-b2ab-5906be021b13 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:51.081935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235d072-3ba3-4854-b2ab-5906be021b13 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:51.082203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235d072-3ba3-4854-b2ab-5906be021b13 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:51.091656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235d072-3ba3-4854-b2ab-5906be021b13 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:51.093160 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e235d072-3ba3-4854-b2ab-5906be021b13 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:51.097410 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920b086-01bd-4057-a527-b10332587266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:51.098201 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920b086-01bd-4057-a527-b10332587266 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:51.098581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920b086-01bd-4057-a527-b10332587266 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:51.099582 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d920b086-01bd-4057-a527-b10332587266 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:51.100323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920b086-01bd-4057-a527-b10332587266 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:51.100628 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920b086-01bd-4057-a527-b10332587266 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:51.102037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d920b086-01bd-4057-a527-b10332587266 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:51.106144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e235d072-3ba3-4854-b2ab-5906be021b13 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:51.106144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:51.106144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:51.106144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:51.106144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:51.106144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:51.106144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:51.106144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:51.106144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:51.106144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:51.106144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:51.106144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:51.106144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:13:51.106144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:13:51.106144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:51.107264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:51.108368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:13:51.108368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:13:51.108368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:13:51.108368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:13:51.108368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:13:51.108368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:13:51.108368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:51.108368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:51.109116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 292/587] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:51.109969 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235d072-3ba3-4854-b2ab-5906be021b13 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:51.119389 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:51.119797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 296/588] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:51.120652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920b086-01bd-4057-a527-b10332587266 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:51.225729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235d072-3ba3-4854-b2ab-5906be021b13 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:51.242596 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d920b086-01bd-4057-a527-b10332587266 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:51.265011 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074de84-fe6d-4c9b-b1bc-98be46941583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:51.265629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074de84-fe6d-4c9b-b1bc-98be46941583 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:51.266405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074de84-fe6d-4c9b-b1bc-98be46941583 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:51.266405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f074de84-fe6d-4c9b-b1bc-98be46941583 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:51.266893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074de84-fe6d-4c9b-b1bc-98be46941583 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:51.267200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074de84-fe6d-4c9b-b1bc-98be46941583 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:51.268708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f074de84-fe6d-4c9b-b1bc-98be46941583 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:51.282446 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2fd3ab-2c4f-4c9e-b20a-600ba1ee904d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:51.283054 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fd3ab-2c4f-4c9e-b20a-600ba1ee904d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:51.283330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fd3ab-2c4f-4c9e-b20a-600ba1ee904d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:51.283596 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2fd3ab-2c4f-4c9e-b20a-600ba1ee904d None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:51.284240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2fd3ab-2c4f-4c9e-b20a-600ba1ee904d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:51.284538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2fd3ab-2c4f-4c9e-b20a-600ba1ee904d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:51.285891 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2fd3ab-2c4f-4c9e-b20a-600ba1ee904d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:51.288235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 293/589] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:13:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:51.289142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074de84-fe6d-4c9b-b1bc-98be46941583 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:51.298729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 297/590] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:51.299771 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2fd3ab-2c4f-4c9e-b20a-600ba1ee904d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:51.338476 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:51.370801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:51.414176 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f074de84-fe6d-4c9b-b1bc-98be46941583 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:51.419552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2fd3ab-2c4f-4c9e-b20a-600ba1ee904d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:51.460810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc18f3-3f5b-4311-92b2-86d5ba9269da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:51.462025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc18f3-3f5b-4311-92b2-86d5ba9269da None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:51.462025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc18f3-3f5b-4311-92b2-86d5ba9269da None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:51.462025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc18f3-3f5b-4311-92b2-86d5ba9269da None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:51.466227 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b73b43-cc1b-4fcc-b264-d9762dd4e3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:51.466929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b73b43-cc1b-4fcc-b264-d9762dd4e3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:51.467320 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b73b43-cc1b-4fcc-b264-d9762dd4e3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:51.467638 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b73b43-cc1b-4fcc-b264-d9762dd4e3d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:51.468411 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b73b43-cc1b-4fcc-b264-d9762dd4e3d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:51.468783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b73b43-cc1b-4fcc-b264-d9762dd4e3d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:51.470304 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b73b43-cc1b-4fcc-b264-d9762dd4e3d4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:51.472754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc18f3-3f5b-4311-92b2-86d5ba9269da None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:51.475217 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc18f3-3f5b-4311-92b2-86d5ba9269da None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:51.490489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 294/591] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:13:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:51.490489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b73b43-cc1b-4fcc-b264-d9762dd4e3d4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:51.490884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-cabc18f3-3f5b-4311-92b2-86d5ba9269da None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 15:13:51.490884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:51.490884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:51.490884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:51.490884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:51.490884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:51.490884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:51.490884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:51.490884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:51.490884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:51.490884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:51.490884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:51.490884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:13:51.490884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:13:51.490884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:13:51.490884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:51.492121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:51.493873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:51.493873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:51.493873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:13:51.493873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:13:51.493873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:13:51.493873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:13:51.493873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:13:51.493873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:13:51.493873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 15:13:51.493873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:51.495100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 298/592] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:51 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:51.495305 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc18f3-3f5b-4311-92b2-86d5ba9269da None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:51.625543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b73b43-cc1b-4fcc-b264-d9762dd4e3d4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:51.632353 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:51.638763 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc18f3-3f5b-4311-92b2-86d5ba9269da None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:51.657122 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:51.671716 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e1c6d-8fd0-43fd-a5e1-bd3f5e4c888b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:51.672598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e1c6d-8fd0-43fd-a5e1-bd3f5e4c888b None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:51.672889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e1c6d-8fd0-43fd-a5e1-bd3f5e4c888b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:51.678209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709e1c6d-8fd0-43fd-a5e1-bd3f5e4c888b None admin] PATH_INFO: `/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/96c71ea88d9a474c9ff7a30555cfa91f/roles/59a2b50693a646fc8332b835233486ce` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:51.689001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d7b2b-c60c-4d14-b8b1-62e58e1fc63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:51.690612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d7b2b-c60c-4d14-b8b1-62e58e1fc63d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:51.691073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d7b2b-c60c-4d14-b8b1-62e58e1fc63d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:51.691527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d7b2b-c60c-4d14-b8b1-62e58e1fc63d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:51.698426 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e1c6d-8fd0-43fd-a5e1-bd3f5e4c888b None admin] RBAC: Authorizing `identity:create_grant(project_id=2c010096b77f4cd3b2f32086b1ba7ea7, user_id=96c71ea88d9a474c9ff7a30555cfa91f, role_id=59a2b50693a646fc8332b835233486ce)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:51.700329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709e1c6d-8fd0-43fd-a5e1-bd3f5e4c888b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:51.703921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d7b2b-c60c-4d14-b8b1-62e58e1fc63d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:51.705845 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d7b2b-c60c-4d14-b8b1-62e58e1fc63d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:51.715866 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 295/593] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:13:51 2022] PUT /identity/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/96c71ea88d9a474c9ff7a30555cfa91f/roles/59a2b50693a646fc8332b835233486ce => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:13:51.716963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e1c6d-8fd0-43fd-a5e1-bd3f5e4c888b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:51.720984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-193d7b2b-c60c-4d14-b8b1-62e58e1fc63d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 15:13:51.720984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:51.720984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:51.720984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:51.720984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:51.720984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:51.720984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:51.720984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:51.720984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:51.720984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:51.720984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:51.720984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:51.720984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:13:51.720984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:13:51.720984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:13:51.720984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:51.722833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:51.724331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:51.724331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:51.724331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:13:51.724331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:13:51.724331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:13:51.724331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:13:51.724331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:13:51.724331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:13:51.724331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 15:13:51.724331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:51.724331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 299/594] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:51 2022] GET /identity/v3/users/cinder => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:51.724331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d7b2b-c60c-4d14-b8b1-62e58e1fc63d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:51.862064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709e1c6d-8fd0-43fd-a5e1-bd3f5e4c888b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:51.909581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d7b2b-c60c-4d14-b8b1-62e58e1fc63d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:51.911264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8617eea-c198-4d9d-b818-30c77fade847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:51.913728 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8617eea-c198-4d9d-b818-30c77fade847 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:51.914054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8617eea-c198-4d9d-b818-30c77fade847 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:51.914389 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8617eea-c198-4d9d-b818-30c77fade847 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:51.915077 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8617eea-c198-4d9d-b818-30c77fade847 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:51.916123 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8617eea-c198-4d9d-b818-30c77fade847 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:51.916813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8617eea-c198-4d9d-b818-30c77fade847 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:51.939943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 296/595] 10.222.0.76 () {56 vars in 1139 bytes} [Wed Aug 24 15:13:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:51.939943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8617eea-c198-4d9d-b818-30c77fade847 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:51.956671 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e35bf-f6d6-455e-b91b-e8638016f9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:51.957273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e35bf-f6d6-455e-b91b-e8638016f9be None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:51.957561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e35bf-f6d6-455e-b91b-e8638016f9be None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:51.957827 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0e35bf-f6d6-455e-b91b-e8638016f9be None admin] PATH_INFO: `/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/8a5fd31a2db944f596912e4f0aa26835/roles/59a2b50693a646fc8332b835233486ce` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:51.979821 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e35bf-f6d6-455e-b91b-e8638016f9be None admin] RBAC: Authorizing `identity:create_grant(project_id=2c010096b77f4cd3b2f32086b1ba7ea7, user_id=8a5fd31a2db944f596912e4f0aa26835, role_id=59a2b50693a646fc8332b835233486ce)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:51.982629 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0e35bf-f6d6-455e-b91b-e8638016f9be None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:51.984393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:52.016802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 300/596] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:13:51 2022] PUT /identity/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/8a5fd31a2db944f596912e4f0aa26835/roles/59a2b50693a646fc8332b835233486ce => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:13:52.016802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e35bf-f6d6-455e-b91b-e8638016f9be None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:52.042171 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:52.113560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8617eea-c198-4d9d-b818-30c77fade847 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:52.196137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77295c-6694-4b92-9e06-9bf18b95489c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:52.196768 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77295c-6694-4b92-9e06-9bf18b95489c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:52.197173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77295c-6694-4b92-9e06-9bf18b95489c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:52.197817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac77295c-6694-4b92-9e06-9bf18b95489c None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:52.198305 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77295c-6694-4b92-9e06-9bf18b95489c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:52.198611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77295c-6694-4b92-9e06-9bf18b95489c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:52.200102 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac77295c-6694-4b92-9e06-9bf18b95489c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:52.203026 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0e35bf-f6d6-455e-b91b-e8638016f9be None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:52.238899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 297/597] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 24 15:13:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:52.244253 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77295c-6694-4b92-9e06-9bf18b95489c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:52.253317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3a8ff-9543-4fab-95f1-de374a6a1b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:52.254657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3a8ff-9543-4fab-95f1-de374a6a1b24 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:52.255776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3a8ff-9543-4fab-95f1-de374a6a1b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:52.256446 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d3a8ff-9543-4fab-95f1-de374a6a1b24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:52.269096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3a8ff-9543-4fab-95f1-de374a6a1b24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:52.271527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d3a8ff-9543-4fab-95f1-de374a6a1b24 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:52.293083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-94d3a8ff-9543-4fab-95f1-de374a6a1b24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:52.293083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:52.293083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:52.293083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:52.293083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:52.293083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:52.293083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:52.293083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:52.293083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:52.293083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:52.293083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:52.293083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:52.293083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:52.293083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:52.293083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:52.293083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:52.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:52.295921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:52.295921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:52.295921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:52.297471 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 301/598] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:52.305459 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3a8ff-9543-4fab-95f1-de374a6a1b24 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:52.383514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:52.390915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:52.396803 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac77295c-6694-4b92-9e06-9bf18b95489c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:52.443547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d3a8ff-9543-4fab-95f1-de374a6a1b24 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:52.450462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09588e40-441b-4d9b-9597-92fc7cfbf71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:52.452026 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09588e40-441b-4d9b-9597-92fc7cfbf71e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:52.452790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09588e40-441b-4d9b-9597-92fc7cfbf71e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:52.453418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09588e40-441b-4d9b-9597-92fc7cfbf71e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:52.472030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09588e40-441b-4d9b-9597-92fc7cfbf71e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:52.474319 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09588e40-441b-4d9b-9597-92fc7cfbf71e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:52.483755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cabab1-5bb5-40b1-b00b-f434e4ea3e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:52.484532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cabab1-5bb5-40b1-b00b-f434e4ea3e61 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:52.484931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cabab1-5bb5-40b1-b00b-f434e4ea3e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:52.485402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cabab1-5bb5-40b1-b00b-f434e4ea3e61 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:52.486135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cabab1-5bb5-40b1-b00b-f434e4ea3e61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:52.486563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cabab1-5bb5-40b1-b00b-f434e4ea3e61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:52.488048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cabab1-5bb5-40b1-b00b-f434e4ea3e61 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:52.493805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-09588e40-441b-4d9b-9597-92fc7cfbf71e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:52.493805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:52.493805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:52.493805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:52.493805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:52.493805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:52.493805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:52.493805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:52.493805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:52.493805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:52.493805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:52.493805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:52.493805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:52.493805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:52.493805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:52.493805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:52.494914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:52.495989 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:52.495989 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:52.495989 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:52.501219 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 298/599] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:52.502905 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013258c-d48a-4603-9a95-890da447edc4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:52.503340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013258c-d48a-4603-9a95-890da447edc4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:52.503756 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013258c-d48a-4603-9a95-890da447edc4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:52.506085 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 299/601] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:52.509453 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013258c-d48a-4603-9a95-890da447edc4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:52.509882 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 302/601] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:52.511750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859e310-363d-4f85-bffa-434cd4086731 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:52.512124 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859e310-363d-4f85-bffa-434cd4086731 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:52.512532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e859e310-363d-4f85-bffa-434cd4086731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:52.539200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e859e310-363d-4f85-bffa-434cd4086731 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:52.541273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e859e310-363d-4f85-bffa-434cd4086731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:52.633688 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 303/602] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:52.635148 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859e310-363d-4f85-bffa-434cd4086731 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:52.635960 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013258c-d48a-4603-9a95-890da447edc4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:52.674452 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaad76-ab04-46ea-98bc-97d9e299d7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:52.675088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaad76-ab04-46ea-98bc-97d9e299d7ff None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:52.675368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaad76-ab04-46ea-98bc-97d9e299d7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:52.675635 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaad76-ab04-46ea-98bc-97d9e299d7ff None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:52.676336 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeaad76-ab04-46ea-98bc-97d9e299d7ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:52.676639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeaad76-ab04-46ea-98bc-97d9e299d7ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:52.678680 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeaad76-ab04-46ea-98bc-97d9e299d7ff None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:52.692412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 300/603] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:52.693584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:52.699638 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaad76-ab04-46ea-98bc-97d9e299d7ff None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:52.705747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:52.757257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e859e310-363d-4f85-bffa-434cd4086731 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:52.820223 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f6fc6-8b85-4b35-95be-aef5536924e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:52.821352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f6fc6-8b85-4b35-95be-aef5536924e3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:52.821943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f6fc6-8b85-4b35-95be-aef5536924e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:52.822591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f6fc6-8b85-4b35-95be-aef5536924e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:52.841378 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaad76-ab04-46ea-98bc-97d9e299d7ff None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:52.842383 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41f6fc6-8b85-4b35-95be-aef5536924e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:52.843984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41f6fc6-8b85-4b35-95be-aef5536924e3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:52.862245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-a41f6fc6-8b85-4b35-95be-aef5536924e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:52.862245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:52.862245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:52.862245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:52.862245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:52.862245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:52.862245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:52.862245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:52.862245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:52.862245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:52.862245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:52.862245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:52.862245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:13:52.862245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:13:52.862245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:52.863183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:52.864032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:13:52.864032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:13:52.864032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:13:52.864032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:13:52.864032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:13:52.864032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:13:52.864032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:52.864032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:52.866528 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 304/604] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:52.884028 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f6fc6-8b85-4b35-95be-aef5536924e3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:52.888797 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd65c1-aed6-4042-8fad-d8ad193d7eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:52.889758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd65c1-aed6-4042-8fad-d8ad193d7eae None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:52.890259 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd65c1-aed6-4042-8fad-d8ad193d7eae None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:52.890721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dd65c1-aed6-4042-8fad-d8ad193d7eae None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:52.909055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd65c1-aed6-4042-8fad-d8ad193d7eae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:52.911082 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dd65c1-aed6-4042-8fad-d8ad193d7eae None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:52.934921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-66dd65c1-aed6-4042-8fad-d8ad193d7eae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:52.934921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:52.934921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:52.934921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:52.934921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:52.934921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:52.934921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:52.934921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:52.934921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:52.934921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:52.934921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:52.934921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:52.934921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:13:52.934921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:13:52.934921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:52.935838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:52.936724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:13:52.936724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:13:52.936724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:13:52.936724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:13:52.936724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:13:52.936724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:13:52.936724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:52.936724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:52.940459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 301/605] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:13:52.946711 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd65c1-aed6-4042-8fad-d8ad193d7eae None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:53.050889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f6fc6-8b85-4b35-95be-aef5536924e3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:53.061700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:53.066082 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:53.093721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b4411-3877-462b-b041-f8a8aa9e1608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:53.094774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b4411-3877-462b-b041-f8a8aa9e1608 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:53.095268 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b4411-3877-462b-b041-f8a8aa9e1608 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:53.095825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b4411-3877-462b-b041-f8a8aa9e1608 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:53.096812 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b4411-3877-462b-b041-f8a8aa9e1608 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:53.097318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b4411-3877-462b-b041-f8a8aa9e1608 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:53.098989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b4411-3877-462b-b041-f8a8aa9e1608 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:53.112149 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 305/606] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:13:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:53.113249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b4411-3877-462b-b041-f8a8aa9e1608 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:53.119301 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dd65c1-aed6-4042-8fad-d8ad193d7eae None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:53.170596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a7d47-882e-440f-9be3-99316fa285c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:53.171215 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a7d47-882e-440f-9be3-99316fa285c0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:53.171497 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a7d47-882e-440f-9be3-99316fa285c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:53.172150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a7d47-882e-440f-9be3-99316fa285c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:53.172874 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a7d47-882e-440f-9be3-99316fa285c0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:53.173182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a7d47-882e-440f-9be3-99316fa285c0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:53.174654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a7d47-882e-440f-9be3-99316fa285c0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:53.194312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 302/607] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:13:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:53.195559 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a7d47-882e-440f-9be3-99316fa285c0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:53.241490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b4411-3877-462b-b041-f8a8aa9e1608 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:53.287554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb90cd-26ce-4b07-bd8b-910045e865b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:53.288196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb90cd-26ce-4b07-bd8b-910045e865b9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:53.288474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb90cd-26ce-4b07-bd8b-910045e865b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:53.288748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb90cd-26ce-4b07-bd8b-910045e865b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:53.302818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb90cd-26ce-4b07-bd8b-910045e865b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:53.309630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-cedb90cd-26ce-4b07-bd8b-910045e865b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:53.309630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:53.309630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:53.309630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:53.309630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:53.309630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:53.309630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:53.309630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:53.309630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:53.309630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:53.309630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:53.309630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:53.309630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:53.309630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:53.309630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:53.309630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:53.310885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:53.310885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:53.310885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:53.310885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:53.310885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:53.310885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:53.310885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:53.310885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:53.310885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:53.310885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:53.310885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:53.310885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:53.310885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:53.310885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:53.310885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:53.310885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:53.312100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:53.312100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:53.312100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:53.312100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:53.312100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:53.312100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:53.312100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:53.312100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:53.312100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:53.312100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:53.312100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:53.312100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:53.312100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 306/608] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:53.312100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb90cd-26ce-4b07-bd8b-910045e865b9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:53.320093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a7d47-882e-440f-9be3-99316fa285c0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:53.348762 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:53.349608 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:53.363925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d760c37-2de9-44b1-9c47-b6abc9deacbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:53.365330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d760c37-2de9-44b1-9c47-b6abc9deacbf None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:53.365967 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d760c37-2de9-44b1-9c47-b6abc9deacbf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:53.366371 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d760c37-2de9-44b1-9c47-b6abc9deacbf None admin] PATH_INFO: `/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/5427bd0ac88042338b956c10cfd07baa/roles/59a2b50693a646fc8332b835233486ce` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:53.383386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d760c37-2de9-44b1-9c47-b6abc9deacbf None admin] RBAC: Authorizing `identity:create_grant(project_id=2c010096b77f4cd3b2f32086b1ba7ea7, user_id=5427bd0ac88042338b956c10cfd07baa, role_id=59a2b50693a646fc8332b835233486ce)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:53.385129 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d760c37-2de9-44b1-9c47-b6abc9deacbf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:53.405518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 303/609] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:13:53 2022] PUT /identity/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/5427bd0ac88042338b956c10cfd07baa/roles/59a2b50693a646fc8332b835233486ce => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:13:53.406770 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d760c37-2de9-44b1-9c47-b6abc9deacbf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:53.492677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb90cd-26ce-4b07-bd8b-910045e865b9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:53.565474 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d760c37-2de9-44b1-9c47-b6abc9deacbf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:53.568190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b3962-b8f9-4920-836c-7e0558ef2d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:53.568936 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b3962-b8f9-4920-836c-7e0558ef2d9c None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:53.569280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b3962-b8f9-4920-836c-7e0558ef2d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:53.569867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b3962-b8f9-4920-836c-7e0558ef2d9c None admin] PATH_INFO: `/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/194c7492cf954a6a8dd21de66aa24fec/roles/59a2b50693a646fc8332b835233486ce` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:53.588928 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b3962-b8f9-4920-836c-7e0558ef2d9c None admin] RBAC: Authorizing `identity:create_grant(project_id=2c010096b77f4cd3b2f32086b1ba7ea7, user_id=194c7492cf954a6a8dd21de66aa24fec, role_id=59a2b50693a646fc8332b835233486ce)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:53.590616 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b3962-b8f9-4920-836c-7e0558ef2d9c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:53.607877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 307/610] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:13:53 2022] PUT /identity/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/194c7492cf954a6a8dd21de66aa24fec/roles/59a2b50693a646fc8332b835233486ce => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:13:53.639072 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d4e66-3043-4672-bb9d-e9e6803c3672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:53.639703 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d4e66-3043-4672-bb9d-e9e6803c3672 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:53.639981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d4e66-3043-4672-bb9d-e9e6803c3672 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:53.640315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d4e66-3043-4672-bb9d-e9e6803c3672 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:53.641115 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d4e66-3043-4672-bb9d-e9e6803c3672 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:53.641444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d4e66-3043-4672-bb9d-e9e6803c3672 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:53.644282 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d4e66-3043-4672-bb9d-e9e6803c3672 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:53.654934 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 304/611] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:13:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:53.660357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b3962-b8f9-4920-836c-7e0558ef2d9c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:53.784088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:53.815787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b3962-b8f9-4920-836c-7e0558ef2d9c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:53.868073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351eb4f4-eb7a-47ab-99d7-e0f5426bae41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:53.868735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351eb4f4-eb7a-47ab-99d7-e0f5426bae41 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:53.869017 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351eb4f4-eb7a-47ab-99d7-e0f5426bae41 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:53.869286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351eb4f4-eb7a-47ab-99d7-e0f5426bae41 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:53.886543 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351eb4f4-eb7a-47ab-99d7-e0f5426bae41 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:53.888015 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351eb4f4-eb7a-47ab-99d7-e0f5426bae41 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:53.909108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-351eb4f4-eb7a-47ab-99d7-e0f5426bae41 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:53.909108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:53.909108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:53.909108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:53.909108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:53.909108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:53.909108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:53.909108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:53.909108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:53.909108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:53.909108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:53.909108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:53.909108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:53.909108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:53.909108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:53.909108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:53.910091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:53.911454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:53.911454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:53.911454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:53.911454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 308/612] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:53.916312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d4e66-3043-4672-bb9d-e9e6803c3672 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:53.965045 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:54.020794 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648663ee-d274-4c8f-97a4-7d12186fb21d None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:54.021160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648663ee-d274-4c8f-97a4-7d12186fb21d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:54.021330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648663ee-d274-4c8f-97a4-7d12186fb21d None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:54.022956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 309/613] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:54.028741 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637d92c-6581-49bd-b310-eb655e850ed1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:54.029023 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637d92c-6581-49bd-b310-eb655e850ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:54.029328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0637d92c-6581-49bd-b310-eb655e850ed1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:54.050006 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d4e66-3043-4672-bb9d-e9e6803c3672 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:54.051057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0637d92c-6581-49bd-b310-eb655e850ed1 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:54.052797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0637d92c-6581-49bd-b310-eb655e850ed1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:54.122516 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec6b3a-40e4-446a-a412-8a2fd9b7c4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:54.123052 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec6b3a-40e4-446a-a412-8a2fd9b7c4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:54.123308 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec6b3a-40e4-446a-a412-8a2fd9b7c4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:54.123581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ec6b3a-40e4-446a-a412-8a2fd9b7c4d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:54.124237 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec6b3a-40e4-446a-a412-8a2fd9b7c4d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:54.124539 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec6b3a-40e4-446a-a412-8a2fd9b7c4d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:54.125927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ec6b3a-40e4-446a-a412-8a2fd9b7c4d8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:54.148864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 305/614] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:54.157203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec6b3a-40e4-446a-a412-8a2fd9b7c4d8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:54.199797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 310/615] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:13:54.287524 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:54.293480 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ec6b3a-40e4-446a-a412-8a2fd9b7c4d8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:54.333858 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb5cdc-ff00-4f86-9514-40bf25a1d255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:54.335002 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb5cdc-ff00-4f86-9514-40bf25a1d255 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:54.335532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb5cdc-ff00-4f86-9514-40bf25a1d255 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:54.336069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fb5cdc-ff00-4f86-9514-40bf25a1d255 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:54.345367 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb5cdc-ff00-4f86-9514-40bf25a1d255 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:54.346933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fb5cdc-ff00-4f86-9514-40bf25a1d255 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:54.359260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f0fb5cdc-ff00-4f86-9514-40bf25a1d255 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:13:54.359260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:54.359260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:54.359260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:54.359260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:54.359260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:54.359260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:54.359260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:54.359260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:54.359260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:54.359260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:54.359260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:54.359260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:13:54.359260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:13:54.359260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:13:54.359260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:54.360390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:54.361550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:54.361550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:54.361550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:13:54.361550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:13:54.361550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:13:54.361550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:13:54.361550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:13:54.361550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:13:54.361550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:13:54.361550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:54.362445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 306/616] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:13:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:54.368934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637d92c-6581-49bd-b310-eb655e850ed1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:54.465949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:54.521272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb5cdc-ff00-4f86-9514-40bf25a1d255 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:54.537705 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0637d92c-6581-49bd-b310-eb655e850ed1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:54.599647 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bbc0e5-cefd-4051-a49e-39a16da7e0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:54.600262 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbc0e5-cefd-4051-a49e-39a16da7e0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:54.600534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbc0e5-cefd-4051-a49e-39a16da7e0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:54.600800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bbc0e5-cefd-4051-a49e-39a16da7e0a4 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:54.601537 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbc0e5-cefd-4051-a49e-39a16da7e0a4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:54.601837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbc0e5-cefd-4051-a49e-39a16da7e0a4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:54.603302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bbc0e5-cefd-4051-a49e-39a16da7e0a4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:54.637607 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 311/617] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 24 15:13:54 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:54.638922 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b068502-f665-4adb-9a3a-009eeb798783 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:54.639215 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b068502-f665-4adb-9a3a-009eeb798783 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:54.639484 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b068502-f665-4adb-9a3a-009eeb798783 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:54.640930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 312/618] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:54.647185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b068502-f665-4adb-9a3a-009eeb798783 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:54.668132 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fb5cdc-ff00-4f86-9514-40bf25a1d255 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:54.716439 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d0d61-0ba3-43f3-8635-6140cd0b2644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:54.717383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d0d61-0ba3-43f3-8635-6140cd0b2644 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:54.717827 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d0d61-0ba3-43f3-8635-6140cd0b2644 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:54.718111 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d0d61-0ba3-43f3-8635-6140cd0b2644 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:54.732488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d0d61-0ba3-43f3-8635-6140cd0b2644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:54.738473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-698d0d61-0ba3-43f3-8635-6140cd0b2644 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:54.738473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:54.738473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:54.738473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:54.738473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:54.738473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:54.738473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:54.738473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:54.738473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:54.738473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:54.738473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:54.738473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:54.738473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:54.738473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:54.738473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:54.738473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:54.740142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:54.740142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:54.740142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:54.740142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:54.740142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:54.740142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:54.740142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:54.740142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:54.740142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:54.740142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:54.740142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:54.740142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:54.740142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:54.740142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:54.740142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:54.740142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:54.745745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:54.745745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:54.745745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:54.745745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:54.745745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:54.745745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:54.745745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:54.745745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:54.745745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:54.745745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:54.745745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:54.745745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:54.745745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 307/619] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:13:54.745745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b0256-03fc-47c0-ba4d-ca47a207958b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:54.746705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b0256-03fc-47c0-ba4d-ca47a207958b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:54.746705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8b0256-03fc-47c0-ba4d-ca47a207958b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:54.766851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6c8b0256-03fc-47c0-ba4d-ca47a207958b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:54.768261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6c8b0256-03fc-47c0-ba4d-ca47a207958b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:54.780761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:54.792416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b068502-f665-4adb-9a3a-009eeb798783 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:54.812155 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:54.832004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735da35-46d1-4ce7-8a51-9cb6ec44ba5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:54.832955 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735da35-46d1-4ce7-8a51-9cb6ec44ba5c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:54.833985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735da35-46d1-4ce7-8a51-9cb6ec44ba5c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:54.834425 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2735da35-46d1-4ce7-8a51-9cb6ec44ba5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:54.845699 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735da35-46d1-4ce7-8a51-9cb6ec44ba5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:54.847755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2735da35-46d1-4ce7-8a51-9cb6ec44ba5c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:54.854744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 308/620] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:54.855956 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b0256-03fc-47c0-ba4d-ca47a207958b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:54.865337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-2735da35-46d1-4ce7-8a51-9cb6ec44ba5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:54.865337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:54.865337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:54.865337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:54.865337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:54.865337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:54.865337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:54.865337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:54.865337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:54.865337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:54.865337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:54.865337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:54.865337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:54.865337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:54.865337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:54.865337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:54.866745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:54.867708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:54.867708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:54.867708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:54.869546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 313/621] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:54.875696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735da35-46d1-4ce7-8a51-9cb6ec44ba5c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:55.061980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2735da35-46d1-4ce7-8a51-9cb6ec44ba5c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:55.068756 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8b0256-03fc-47c0-ba4d-ca47a207958b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:55.133800 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc326554-d4ad-4d8b-a87c-de6bc80ca048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:55.134593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc326554-d4ad-4d8b-a87c-de6bc80ca048 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:55.134875 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc326554-d4ad-4d8b-a87c-de6bc80ca048 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:55.135171 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc326554-d4ad-4d8b-a87c-de6bc80ca048 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:55.135916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc326554-d4ad-4d8b-a87c-de6bc80ca048 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:55.136240 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc326554-d4ad-4d8b-a87c-de6bc80ca048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:55.137776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc326554-d4ad-4d8b-a87c-de6bc80ca048 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:55.149764 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 309/622] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:13:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:55.155889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc326554-d4ad-4d8b-a87c-de6bc80ca048 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:55.165934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8352348-6f90-4381-b5c5-661f1c6eba67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:55.166663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8352348-6f90-4381-b5c5-661f1c6eba67 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:55.166965 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8352348-6f90-4381-b5c5-661f1c6eba67 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:55.167327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8352348-6f90-4381-b5c5-661f1c6eba67 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:55.168373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8352348-6f90-4381-b5c5-661f1c6eba67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:55.168770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8352348-6f90-4381-b5c5-661f1c6eba67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:55.173735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8352348-6f90-4381-b5c5-661f1c6eba67 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:55.176190 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:55.186010 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 314/623] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:13:55.191472 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8352348-6f90-4381-b5c5-661f1c6eba67 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:55.236730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:55.337036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc326554-d4ad-4d8b-a87c-de6bc80ca048 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:55.340639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8352348-6f90-4381-b5c5-661f1c6eba67 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:55.381879 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55649f5-d91f-43c0-9001-e6faa8f90f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:55.382547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ea0bd-efb4-4a12-a528-50d0624f7182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:55.383010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55649f5-d91f-43c0-9001-e6faa8f90f03 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:55.383010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55649f5-d91f-43c0-9001-e6faa8f90f03 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:55.383177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55649f5-d91f-43c0-9001-e6faa8f90f03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:55.384374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ea0bd-efb4-4a12-a528-50d0624f7182 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:55.384876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ea0bd-efb4-4a12-a528-50d0624f7182 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:55.385348 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ea0bd-efb4-4a12-a528-50d0624f7182 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:55.393011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ea0bd-efb4-4a12-a528-50d0624f7182 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:55.394790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ea0bd-efb4-4a12-a528-50d0624f7182 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:55.397324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55649f5-d91f-43c0-9001-e6faa8f90f03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:55.399354 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55649f5-d91f-43c0-9001-e6faa8f90f03 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:55.410130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-598ea0bd-efb4-4a12-a528-50d0624f7182 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:55.410130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:55.410130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:55.410130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:55.410130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:55.410130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:55.410130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:55.410130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:55.410130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:55.410130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:55.410130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:55.410130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:55.410130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:13:55.410130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:13:55.410130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:55.411196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:55.412136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:13:55.412136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:13:55.412136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:13:55.412136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:13:55.412136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:13:55.412136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:13:55.412136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:55.412136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:55.414580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 315/624] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:13:55.415758 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ea0bd-efb4-4a12-a528-50d0624f7182 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:55.430872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e55649f5-d91f-43c0-9001-e6faa8f90f03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:55.430872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:55.430872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:55.430872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:55.430872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:55.430872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:55.430872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:55.430872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:55.430872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:55.430872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:55.430872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:55.430872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:55.430872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:55.430872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:55.430872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:55.430872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:55.431886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:55.432894 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:55.432894 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:55.432894 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:55.434388 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 310/625] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:13:55.435506 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55649f5-d91f-43c0-9001-e6faa8f90f03 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:55.538737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:55.555027 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:55.562152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55649f5-d91f-43c0-9001-e6faa8f90f03 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:55.570710 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ea0bd-efb4-4a12-a528-50d0624f7182 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:55.603950 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934d179-1811-4805-936b-09666d2596d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:55.604827 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934d179-1811-4805-936b-09666d2596d7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:55.605273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934d179-1811-4805-936b-09666d2596d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:55.605701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934d179-1811-4805-936b-09666d2596d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:55.606615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934d179-1811-4805-936b-09666d2596d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:55.607126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934d179-1811-4805-936b-09666d2596d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:55.608866 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934d179-1811-4805-936b-09666d2596d7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:55.620361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0b684-f754-4209-99fc-20a415513594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:55.621154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0b684-f754-4209-99fc-20a415513594 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:55.622013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0b684-f754-4209-99fc-20a415513594 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:55.622329 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a0b684-f754-4209-99fc-20a415513594 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:55.626079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 311/626] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:13:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:55.627322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934d179-1811-4805-936b-09666d2596d7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:55.643240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a0b684-f754-4209-99fc-20a415513594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:55.650442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-c7a0b684-f754-4209-99fc-20a415513594 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:55.650442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:55.650442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:55.650442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:55.650442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:55.650442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:55.650442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:55.650442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:55.650442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:55.650442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:55.650442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:55.650442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:55.650442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:55.650442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:55.650442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:55.650442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:55.651661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:55.651661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:55.651661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:55.651661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:55.651661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:55.651661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:55.651661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:55.651661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:55.651661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:55.651661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:55.651661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:55.651661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:55.651661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:55.651661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:55.651661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:55.651661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:55.657560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:55.657560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:55.657560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:55.657560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:55.657560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:55.657560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:55.657560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:55.657560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:55.657560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:55.657560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:55.657560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:55.657560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:55.657560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 316/627] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:55.657560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0b684-f754-4209-99fc-20a415513594 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:55.771182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934d179-1811-4805-936b-09666d2596d7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:55.796741 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a0b684-f754-4209-99fc-20a415513594 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:55.798066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:55.830514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8e657-f7a4-4b9b-b148-c8ef323373ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:55.831973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8e657-f7a4-4b9b-b148-c8ef323373ab None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:55.832751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8e657-f7a4-4b9b-b148-c8ef323373ab None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:55.834038 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc8e657-f7a4-4b9b-b148-c8ef323373ab None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:55.835495 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8e657-f7a4-4b9b-b148-c8ef323373ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:55.836230 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8e657-f7a4-4b9b-b148-c8ef323373ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:55.838631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc8e657-f7a4-4b9b-b148-c8ef323373ab None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:55.846476 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844965a-2f74-40d4-be26-995df05e2f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:55.847891 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844965a-2f74-40d4-be26-995df05e2f53 None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:55.848284 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844965a-2f74-40d4-be26-995df05e2f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:55.848841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844965a-2f74-40d4-be26-995df05e2f53 None admin] PATH_INFO: `/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/0679072b37db41e880a95b738bea47ea/roles/59a2b50693a646fc8332b835233486ce` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:55.856497 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 312/628] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:55.857762 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8e657-f7a4-4b9b-b148-c8ef323373ab None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:55.867097 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844965a-2f74-40d4-be26-995df05e2f53 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c010096b77f4cd3b2f32086b1ba7ea7, user_id=0679072b37db41e880a95b738bea47ea, role_id=59a2b50693a646fc8332b835233486ce)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:55.868726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844965a-2f74-40d4-be26-995df05e2f53 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:55.878987 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:55.883841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 317/629] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:13:55 2022] PUT /identity/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/0679072b37db41e880a95b738bea47ea/roles/59a2b50693a646fc8332b835233486ce => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:13:55.885155 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844965a-2f74-40d4-be26-995df05e2f53 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:56.006790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc8e657-f7a4-4b9b-b148-c8ef323373ab None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:56.024377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844965a-2f74-40d4-be26-995df05e2f53 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:56.063941 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8f5ec2-0ffb-4b7f-8888-b2c19e4465ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:56.064608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f5ec2-0ffb-4b7f-8888-b2c19e4465ae None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:56.064907 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f5ec2-0ffb-4b7f-8888-b2c19e4465ae None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:56.065226 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8f5ec2-0ffb-4b7f-8888-b2c19e4465ae None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:56.065896 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f5ec2-0ffb-4b7f-8888-b2c19e4465ae None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:56.066194 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f5ec2-0ffb-4b7f-8888-b2c19e4465ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:56.068125 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8f5ec2-0ffb-4b7f-8888-b2c19e4465ae None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:56.069240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1f679-d986-4a3a-baa2-902a0751038f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:56.070033 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1f679-d986-4a3a-baa2-902a0751038f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:56.070643 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1f679-d986-4a3a-baa2-902a0751038f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:56.070964 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac1f679-d986-4a3a-baa2-902a0751038f None admin] PATH_INFO: `/v3/users/glance` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:56.078885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 313/630] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:13:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:56.080381 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bd369-03ff-49d6-ac2a-f6f49a755dc4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:56.080664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bd369-03ff-49d6-ac2a-f6f49a755dc4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:56.080935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bd369-03ff-49d6-ac2a-f6f49a755dc4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:56.082257 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 314/631] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:56 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:56.084588 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1f679-d986-4a3a-baa2-902a0751038f None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:56.086104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac1f679-d986-4a3a-baa2-902a0751038f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:56.089136 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc5655-8552-4174-9ddb-de3b1d9690cb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:56.089265 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc5655-8552-4174-9ddb-de3b1d9690cb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:56.089667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fc5655-8552-4174-9ddb-de3b1d9690cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:56.104418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-cac1f679-d986-4a3a-baa2-902a0751038f None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 15:13:56.104418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:56.104418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:56.104418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:56.104418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:56.104418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:56.104418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:56.104418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:56.104418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:56.104418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:56.104418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:56.104418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:56.104418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:13:56.104418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:13:56.104418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:13:56.104418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:56.105751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:56.107032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:56.107032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:56.107032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:13:56.107032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:13:56.107032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:13:56.107032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:13:56.107032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:13:56.107032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:13:56.107032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 15:13:56.107032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:56.109389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 318/632] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:56.110511 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1f679-d986-4a3a-baa2-902a0751038f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:56.112556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-35fc5655-8552-4174-9ddb-de3b1d9690cb None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:56.114182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-35fc5655-8552-4174-9ddb-de3b1d9690cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:56.159808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:56.182473 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:56.195467 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 315/633] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:13:56.196717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8f306-7d7a-437b-a3be-d7b889b1fd20 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:56.196997 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8f306-7d7a-437b-a3be-d7b889b1fd20 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:56.197267 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a8f306-7d7a-437b-a3be-d7b889b1fd20 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:56.202259 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 316/634] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:56.202259 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8f306-7d7a-437b-a3be-d7b889b1fd20 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:56.241660 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac1f679-d986-4a3a-baa2-902a0751038f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:56.286176 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8494d1-d182-402f-99c6-58afd49c69d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:56.287012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8494d1-d182-402f-99c6-58afd49c69d1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:56.287861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8494d1-d182-402f-99c6-58afd49c69d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:56.288195 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8494d1-d182-402f-99c6-58afd49c69d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:56.305000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8494d1-d182-402f-99c6-58afd49c69d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:56.307481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8494d1-d182-402f-99c6-58afd49c69d1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:56.313689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a8f306-7d7a-437b-a3be-d7b889b1fd20 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:56.330190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-da8494d1-d182-402f-99c6-58afd49c69d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:56.330190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:56.330190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:56.330190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:56.330190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:56.330190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:56.330190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:56.330190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:56.330190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:56.330190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:56.330190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:56.330190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:56.330190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:56.330190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:56.330190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:56.330190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:56.331182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:56.332054 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:56.332054 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:56.332054 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:56.332054 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 319/635] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:56.334452 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47e386-857f-4b5d-b4a9-d9f29f7fb273 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:56.334706 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47e386-857f-4b5d-b4a9-d9f29f7fb273 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:56.335093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e47e386-857f-4b5d-b4a9-d9f29f7fb273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:56.352714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c5200-a449-4396-916e-8f437d0d1096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:56.353431 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c5200-a449-4396-916e-8f437d0d1096 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:56.353806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c5200-a449-4396-916e-8f437d0d1096 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:56.354184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5c5200-a449-4396-916e-8f437d0d1096 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:56.355017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c5200-a449-4396-916e-8f437d0d1096 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:56.355383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c5200-a449-4396-916e-8f437d0d1096 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:56.356867 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5c5200-a449-4396-916e-8f437d0d1096 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:56.358425 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3e47e386-857f-4b5d-b4a9-d9f29f7fb273 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:56.361012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3e47e386-857f-4b5d-b4a9-d9f29f7fb273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:56.378448 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 317/636] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 24 15:13:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:56.379472 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c5200-a449-4396-916e-8f437d0d1096 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:56.421124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:56.460376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 320/637] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:56.461655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47e386-857f-4b5d-b4a9-d9f29f7fb273 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:56.507869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5c5200-a449-4396-916e-8f437d0d1096 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:56.508231 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:56.557815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd928b4d-5d7d-4776-a37d-601280418f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:56.558470 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd928b4d-5d7d-4776-a37d-601280418f97 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:56.558793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd928b4d-5d7d-4776-a37d-601280418f97 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:56.559199 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd928b4d-5d7d-4776-a37d-601280418f97 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:56.559893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd928b4d-5d7d-4776-a37d-601280418f97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:56.560224 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd928b4d-5d7d-4776-a37d-601280418f97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:56.561726 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd928b4d-5d7d-4776-a37d-601280418f97 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:56.575507 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 318/638] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:56.576350 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd928b4d-5d7d-4776-a37d-601280418f97 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:56.578978 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e47e386-857f-4b5d-b4a9-d9f29f7fb273 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:56.620167 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e856856-9cd3-4aa8-a5c9-f0ae33a44ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:56.621189 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e856856-9cd3-4aa8-a5c9-f0ae33a44ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:56.621917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e856856-9cd3-4aa8-a5c9-f0ae33a44ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:56.622543 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e856856-9cd3-4aa8-a5c9-f0ae33a44ec2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:56.706453 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:56.725429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd928b4d-5d7d-4776-a37d-601280418f97 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:56.760107 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e856856-9cd3-4aa8-a5c9-f0ae33a44ec2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:56.761565 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e856856-9cd3-4aa8-a5c9-f0ae33a44ec2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:56.796420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745db78-26b5-42d0-aefa-7611e5e52bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:56.797053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745db78-26b5-42d0-aefa-7611e5e52bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:56.797325 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745db78-26b5-42d0-aefa-7611e5e52bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:56.797593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c745db78-26b5-42d0-aefa-7611e5e52bb9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:56.821371 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c745db78-26b5-42d0-aefa-7611e5e52bb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:56.827822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c745db78-26b5-42d0-aefa-7611e5e52bb9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:56.827822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:56.827822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:56.827822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:56.827822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:56.827822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:56.827822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:56.827822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:56.827822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:56.827822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:56.827822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:56.827822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:56.827822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:56.827822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:56.827822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:56.827822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:56.829268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:56.829268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:56.829268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:56.829268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:56.829268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:56.829268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:56.829268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:56.829268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:56.829268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:56.829268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:56.829268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:56.829268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:56.829268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:56.829268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:56.829268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:56.829268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:56.830315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:56.830315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:56.830315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:56.830315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:56.830315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:56.830315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:56.830315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:56.830315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:56.830315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:56.830315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:56.830315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:56.830315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:56.830315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 319/639] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:56.830315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745db78-26b5-42d0-aefa-7611e5e52bb9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:56.846665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:56.922697 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 321/640] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:56.923915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e856856-9cd3-4aa8-a5c9-f0ae33a44ec2 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:56.980840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c745db78-26b5-42d0-aefa-7611e5e52bb9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:57.023729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d655a-4a98-4e98-a616-57d9f65c09a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:57.024500 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d655a-4a98-4e98-a616-57d9f65c09a5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:57.024913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d655a-4a98-4e98-a616-57d9f65c09a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:57.025207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46d655a-4a98-4e98-a616-57d9f65c09a5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:57.034914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d655a-4a98-4e98-a616-57d9f65c09a5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:57.036402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46d655a-4a98-4e98-a616-57d9f65c09a5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:57.051314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f46d655a-4a98-4e98-a616-57d9f65c09a5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:13:57.051314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:57.051314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:57.051314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:57.051314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:57.051314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:57.051314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:57.051314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:57.051314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:57.051314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:57.051314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:57.051314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:57.051314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:13:57.051314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:13:57.051314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:13:57.051314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:57.052871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:57.053789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:57.053789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:57.053789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:13:57.053789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:13:57.053789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:13:57.053789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:13:57.053789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:13:57.053789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:13:57.053789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:13:57.053789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:57.053789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 320/641] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 15:13:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:57.054416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d655a-4a98-4e98-a616-57d9f65c09a5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:57.069129 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:57.069453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e856856-9cd3-4aa8-a5c9-f0ae33a44ec2 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:57.111624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fe101-639f-4051-979c-f65cf967b6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:57.112255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fe101-639f-4051-979c-f65cf967b6ba None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:57.112534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fe101-639f-4051-979c-f65cf967b6ba None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:57.112805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fe101-639f-4051-979c-f65cf967b6ba None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:57.124637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0fe101-639f-4051-979c-f65cf967b6ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:57.129903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-1f0fe101-639f-4051-979c-f65cf967b6ba None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:57.129903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:57.129903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:57.129903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:57.129903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:57.129903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:57.129903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:57.129903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:57.129903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:57.129903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:57.129903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:57.129903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:57.129903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:57.129903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:57.129903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:57.129903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:57.131802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:57.131802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:57.131802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:57.131802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:57.131802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:57.131802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:57.131802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:57.131802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:57.131802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:57.131802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:57.131802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:57.131802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:57.131802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:57.131802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:57.131802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:57.131802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:57.133044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:57.133044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:57.133044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:57.133044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:57.133044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:57.133044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:57.133044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:57.133044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:57.133044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:57.133044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:57.133044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:57.133044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:57.133044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 322/642] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:57.133044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fe101-639f-4051-979c-f65cf967b6ba None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:57.138197 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:57.171553 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46d655a-4a98-4e98-a616-57d9f65c09a5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:57.209483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539498d1-f78b-44a7-a18d-d7932e654c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:57.210706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539498d1-f78b-44a7-a18d-d7932e654c82 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:57.211228 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539498d1-f78b-44a7-a18d-d7932e654c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:57.211786 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539498d1-f78b-44a7-a18d-d7932e654c82 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:57.212999 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539498d1-f78b-44a7-a18d-d7932e654c82 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:57.214272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539498d1-f78b-44a7-a18d-d7932e654c82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:57.216968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539498d1-f78b-44a7-a18d-d7932e654c82 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:57.229131 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 321/643] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:13:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:57.229989 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539498d1-f78b-44a7-a18d-d7932e654c82 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:57.253014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fe101-639f-4051-979c-f65cf967b6ba None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:57.290661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1c704-3457-4024-b572-d731d5e5da3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:57.291412 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1c704-3457-4024-b572-d731d5e5da3f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:57.291696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1c704-3457-4024-b572-d731d5e5da3f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:57.292029 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb1c704-3457-4024-b572-d731d5e5da3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:57.303883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb1c704-3457-4024-b572-d731d5e5da3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:57.305330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb1c704-3457-4024-b572-d731d5e5da3f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:57.307312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:57.323754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-bbb1c704-3457-4024-b572-d731d5e5da3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:57.323754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:57.323754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:57.323754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:57.323754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:57.323754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:57.323754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:57.323754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:57.323754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:57.323754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:57.323754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:57.323754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:57.323754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:57.323754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:57.323754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:57.323754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:57.324930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:57.325988 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:57.325988 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:57.325988 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:57.325988 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 323/644] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:57.325988 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1c704-3457-4024-b572-d731d5e5da3f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:57.348529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539498d1-f78b-44a7-a18d-d7932e654c82 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:57.386206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace76905-6ce3-4667-9231-78eee0826d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:57.386835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace76905-6ce3-4667-9231-78eee0826d68 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:57.387107 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace76905-6ce3-4667-9231-78eee0826d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:57.387371 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace76905-6ce3-4667-9231-78eee0826d68 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:57.388160 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace76905-6ce3-4667-9231-78eee0826d68 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:57.388460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace76905-6ce3-4667-9231-78eee0826d68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:57.388589 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:57.389892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace76905-6ce3-4667-9231-78eee0826d68 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:57.409603 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 322/645] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 15:13:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:57.410267 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace76905-6ce3-4667-9231-78eee0826d68 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:57.452378 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb1c704-3457-4024-b572-d731d5e5da3f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:57.500000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb0977-1c47-4c75-9056-f4757f50540d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:57.500975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb0977-1c47-4c75-9056-f4757f50540d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:57.501482 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb0977-1c47-4c75-9056-f4757f50540d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:57.502013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb0977-1c47-4c75-9056-f4757f50540d None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:57.503069 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb0977-1c47-4c75-9056-f4757f50540d None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:57.503642 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb0977-1c47-4c75-9056-f4757f50540d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:57.506022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb0977-1c47-4c75-9056-f4757f50540d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:57.518296 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 324/646] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:13:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:57.519705 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb0977-1c47-4c75-9056-f4757f50540d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:57.558919 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace76905-6ce3-4667-9231-78eee0826d68 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:57.589742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:57.596961 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84800bac-5a84-43ba-b85c-227acb9609bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:57.597597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84800bac-5a84-43ba-b85c-227acb9609bc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:57.597877 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84800bac-5a84-43ba-b85c-227acb9609bc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:57.598146 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84800bac-5a84-43ba-b85c-227acb9609bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:57.609777 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84800bac-5a84-43ba-b85c-227acb9609bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:57.611206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84800bac-5a84-43ba-b85c-227acb9609bc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:57.626860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-84800bac-5a84-43ba-b85c-227acb9609bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:57.626860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:57.626860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:57.626860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:57.626860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:57.626860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:57.626860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:57.626860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:57.626860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:57.626860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:57.626860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:57.626860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:57.626860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:57.626860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:57.626860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:57.626860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:57.627812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:57.633840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:57.633840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:57.633840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:57.633840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 323/647] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:57.633840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84800bac-5a84-43ba-b85c-227acb9609bc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:57.634207 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb0977-1c47-4c75-9056-f4757f50540d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:57.650507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:57.675076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3bb25-540d-4ccf-907b-6e0536220fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:57.675887 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3bb25-540d-4ccf-907b-6e0536220fea None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:57.676258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3bb25-540d-4ccf-907b-6e0536220fea None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:57.676654 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3bb25-540d-4ccf-907b-6e0536220fea None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:57.677937 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3bb25-540d-4ccf-907b-6e0536220fea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:57.678536 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3bb25-540d-4ccf-907b-6e0536220fea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:57.681019 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3bb25-540d-4ccf-907b-6e0536220fea None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:57.694592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 325/648] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:57.695445 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3bb25-540d-4ccf-907b-6e0536220fea None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:57.760020 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84800bac-5a84-43ba-b85c-227acb9609bc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:57.806844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857efab1-512e-4f5c-8e7e-d27d4abd1dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:57.807975 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857efab1-512e-4f5c-8e7e-d27d4abd1dd5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:57.808636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857efab1-512e-4f5c-8e7e-d27d4abd1dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:57.809118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857efab1-512e-4f5c-8e7e-d27d4abd1dd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:57.818854 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3bb25-540d-4ccf-907b-6e0536220fea None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:57.857064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8145f-61c6-4c1c-a683-2da8aadba8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:57.857698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8145f-61c6-4c1c-a683-2da8aadba8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:57.857979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8145f-61c6-4c1c-a683-2da8aadba8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:57.858275 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8145f-61c6-4c1c-a683-2da8aadba8d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:57.868634 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:57.868915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8145f-61c6-4c1c-a683-2da8aadba8d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:57.870394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8145f-61c6-4c1c-a683-2da8aadba8d0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:57.888034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-b6f8145f-61c6-4c1c-a683-2da8aadba8d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:57.888034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:57.888034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:57.888034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:57.888034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:57.888034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:57.888034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:57.888034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:57.888034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:57.888034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:57.888034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:57.888034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:57.888034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:57.888034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:57.888034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:57.888034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:57.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:57.889921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:57.889921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:57.889921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:57.889921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 326/649] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:57.891121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8145f-61c6-4c1c-a683-2da8aadba8d0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:57.896826 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857efab1-512e-4f5c-8e7e-d27d4abd1dd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:57.898337 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857efab1-512e-4f5c-8e7e-d27d4abd1dd5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:57.925605 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:58.007007 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8145f-61c6-4c1c-a683-2da8aadba8d0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:58.014326 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 324/650] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:58.015114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857efab1-512e-4f5c-8e7e-d27d4abd1dd5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:58.045528 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821291df-2c7d-40b9-b54c-ab3bbf6f5783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:58.046123 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821291df-2c7d-40b9-b54c-ab3bbf6f5783 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:58.046426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821291df-2c7d-40b9-b54c-ab3bbf6f5783 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:58.046683 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821291df-2c7d-40b9-b54c-ab3bbf6f5783 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:58.047317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821291df-2c7d-40b9-b54c-ab3bbf6f5783 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:58.047614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821291df-2c7d-40b9-b54c-ab3bbf6f5783 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:58.049002 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821291df-2c7d-40b9-b54c-ab3bbf6f5783 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:58.061072 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 327/651] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:58.061874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821291df-2c7d-40b9-b54c-ab3bbf6f5783 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:58.104092 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:58.122890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857efab1-512e-4f5c-8e7e-d27d4abd1dd5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:58.148949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:58.159662 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952583a9-0057-4469-b468-33128da64538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:58.160283 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952583a9-0057-4469-b468-33128da64538 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:58.160555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952583a9-0057-4469-b468-33128da64538 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:58.160822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952583a9-0057-4469-b468-33128da64538 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:58.162158 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821291df-2c7d-40b9-b54c-ab3bbf6f5783 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:58.167943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952583a9-0057-4469-b468-33128da64538 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:58.169417 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952583a9-0057-4469-b468-33128da64538 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:58.180759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-952583a9-0057-4469-b468-33128da64538 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:58.180759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:58.180759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:58.180759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:58.180759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:58.180759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:58.180759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:58.180759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:58.180759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:58.180759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:58.180759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:58.180759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:58.180759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:13:58.180759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:13:58.180759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:58.182195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:58.183109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:13:58.183109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:13:58.183109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:13:58.183109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:13:58.183109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:13:58.183109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:13:58.183109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:58.183109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:58.183109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 325/652] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:58.183109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ec7a3-8b00-47d1-a4b6-acd59e61dae6 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:58.183645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ec7a3-8b00-47d1-a4b6-acd59e61dae6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:58.183645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ec7a3-8b00-47d1-a4b6-acd59e61dae6 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:58.187596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 326/653] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:13:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:13:58.187596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ec7a3-8b00-47d1-a4b6-acd59e61dae6 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:58.200252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693182a7-fbda-452b-9bfd-2bda657aceb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:58.200859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693182a7-fbda-452b-9bfd-2bda657aceb8 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:58.201135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693182a7-fbda-452b-9bfd-2bda657aceb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:58.201400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693182a7-fbda-452b-9bfd-2bda657aceb8 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:58.202034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693182a7-fbda-452b-9bfd-2bda657aceb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:58.202368 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693182a7-fbda-452b-9bfd-2bda657aceb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:58.203722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693182a7-fbda-452b-9bfd-2bda657aceb8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:58.219877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 328/654] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:58.220845 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693182a7-fbda-452b-9bfd-2bda657aceb8 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:58.292799 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ec7a3-8b00-47d1-a4b6-acd59e61dae6 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:58.322415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693182a7-fbda-452b-9bfd-2bda657aceb8 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:58.326917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5a20a-f634-45bf-923c-c051aff1cebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:58.327546 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5a20a-f634-45bf-923c-c051aff1cebb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:58.327827 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5a20a-f634-45bf-923c-c051aff1cebb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:58.328095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5a20a-f634-45bf-923c-c051aff1cebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:58.338839 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5a20a-f634-45bf-923c-c051aff1cebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:58.340322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5a20a-f634-45bf-923c-c051aff1cebb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:58.342166 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:58.357330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d0b5a20a-f634-45bf-923c-c051aff1cebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:58.357330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:58.357330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:58.357330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:58.357330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:58.357330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:58.357330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:58.357330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:58.357330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:58.357330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:58.357330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:58.357330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:58.357330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:13:58.357330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:13:58.357330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:13:58.357330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:58.358364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:13:58.359235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:13:58.359235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:13:58.359235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:58.359235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 327/655] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:58.359235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5a20a-f634-45bf-923c-c051aff1cebb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:58.363053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b910b0d-3ef6-4957-8694-e672e015c041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:58.363477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b910b0d-3ef6-4957-8694-e672e015c041 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:58.363751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b910b0d-3ef6-4957-8694-e672e015c041 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:58.364013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b910b0d-3ef6-4957-8694-e672e015c041 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:58.371806 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b910b0d-3ef6-4957-8694-e672e015c041 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:58.373403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b910b0d-3ef6-4957-8694-e672e015c041 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:58.389221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-0b910b0d-3ef6-4957-8694-e672e015c041 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 15:13:58.389221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:58.389221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:58.389221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:58.389221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:58.389221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:58.389221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:58.389221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:58.389221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:58.389221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:58.389221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:58.389221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:58.389221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:13:58.389221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:13:58.389221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:13:58.389221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:58.390875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:58.392397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:58.392397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:58.392397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:13:58.392397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:13:58.392397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:13:58.392397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:13:58.392397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:13:58.392397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:13:58.392397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 15:13:58.392397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:58.392397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 329/656] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:58.392397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9be51e-c937-4af4-a204-48b916533e0a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:58.392397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9be51e-c937-4af4-a204-48b916533e0a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:58.393110 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9be51e-c937-4af4-a204-48b916533e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:58.410986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5e9be51e-c937-4af4-a204-48b916533e0a None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:13:58.412302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5e9be51e-c937-4af4-a204-48b916533e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:13:58.414367 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:58.471038 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5a20a-f634-45bf-923c-c051aff1cebb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:58.481567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 330/657] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:13:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:13:58.482433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9be51e-c937-4af4-a204-48b916533e0a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:58.507751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae66b1e-c8bb-4ae9-aa8a-3f1eee15aa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:58.508429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae66b1e-c8bb-4ae9-aa8a-3f1eee15aa1c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:58.508698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae66b1e-c8bb-4ae9-aa8a-3f1eee15aa1c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:58.508974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae66b1e-c8bb-4ae9-aa8a-3f1eee15aa1c None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:58.509673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae66b1e-c8bb-4ae9-aa8a-3f1eee15aa1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:58.509980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae66b1e-c8bb-4ae9-aa8a-3f1eee15aa1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:58.511438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae66b1e-c8bb-4ae9-aa8a-3f1eee15aa1c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:58.526812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 328/658] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:13:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:58.527955 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae66b1e-c8bb-4ae9-aa8a-3f1eee15aa1c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:58.622826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9be51e-c937-4af4-a204-48b916533e0a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:58.628718 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:58.672798 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f07a07-2d58-4e59-90b5-5bccf483f090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:58.673428 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f07a07-2d58-4e59-90b5-5bccf483f090 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:58.673703 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f07a07-2d58-4e59-90b5-5bccf483f090 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:58.673970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f07a07-2d58-4e59-90b5-5bccf483f090 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:58.681610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f07a07-2d58-4e59-90b5-5bccf483f090 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:58.683097 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f07a07-2d58-4e59-90b5-5bccf483f090 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:58.686029 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:58.698618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae66b1e-c8bb-4ae9-aa8a-3f1eee15aa1c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:58.702750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-82f07a07-2d58-4e59-90b5-5bccf483f090 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 15:13:58.702750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:58.702750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:58.702750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:58.702750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:58.702750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:58.702750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:58.702750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:58.702750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:58.702750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:58.702750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:58.702750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:58.702750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:13:58.702750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:13:58.702750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:13:58.702750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:58.704704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:58.706737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:58.706737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:58.706737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:13:58.706737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:13:58.706737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:13:58.706737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:13:58.706737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:13:58.706737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:13:58.706737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 15:13:58.706737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:58.711385 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 331/659] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:13:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:58.712458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f07a07-2d58-4e59-90b5-5bccf483f090 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:58.779835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e68426-7f3c-44de-b8db-7c95bb9788c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:58.780719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e68426-7f3c-44de-b8db-7c95bb9788c0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:58.781012 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e68426-7f3c-44de-b8db-7c95bb9788c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:58.781285 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e68426-7f3c-44de-b8db-7c95bb9788c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:58.781966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e68426-7f3c-44de-b8db-7c95bb9788c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:58.782286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e68426-7f3c-44de-b8db-7c95bb9788c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:58.783672 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e68426-7f3c-44de-b8db-7c95bb9788c0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:58.797080 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 329/660] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:58.797994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e68426-7f3c-44de-b8db-7c95bb9788c0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:58.835642 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f07a07-2d58-4e59-90b5-5bccf483f090 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:58.876533 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51850954-97a1-482b-8483-f5bd61b528dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:58.877327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51850954-97a1-482b-8483-f5bd61b528dd None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:58.877710 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51850954-97a1-482b-8483-f5bd61b528dd None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:58.878121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51850954-97a1-482b-8483-f5bd61b528dd None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:58.879221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51850954-97a1-482b-8483-f5bd61b528dd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:58.879697 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51850954-97a1-482b-8483-f5bd61b528dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:58.882499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51850954-97a1-482b-8483-f5bd61b528dd None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:58.914540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 332/661] 10.222.0.76 () {56 vars in 1139 bytes} [Wed Aug 24 15:13:58 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:58.914540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51850954-97a1-482b-8483-f5bd61b528dd None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:58.919576 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e68426-7f3c-44de-b8db-7c95bb9788c0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:58.941852 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:58.966626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3876ea-121f-4f7e-8e5d-2099ff722cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:58.967124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3876ea-121f-4f7e-8e5d-2099ff722cdb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:58.967390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3876ea-121f-4f7e-8e5d-2099ff722cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:58.968279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3876ea-121f-4f7e-8e5d-2099ff722cdb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:58.968407 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3876ea-121f-4f7e-8e5d-2099ff722cdb None admin] RBAC: Adding query filter params (role.id=59a2b50693a646fc8332b835233486ce, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=96c71ea88d9a474c9ff7a30555cfa91f) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:58.968675 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3876ea-121f-4f7e-8e5d-2099ff722cdb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:58.972388 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3876ea-121f-4f7e-8e5d-2099ff722cdb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:58.977218 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 330/662] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:13:58 2022] GET /identity/v3/role_assignments?user.id=96c71ea88d9a474c9ff7a30555cfa91f&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=59a2b50693a646fc8332b835233486ce => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:58.977916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3876ea-121f-4f7e-8e5d-2099ff722cdb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:58.984719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:59.047486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51850954-97a1-482b-8483-f5bd61b528dd None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:59.102919 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794d7aa-ea5d-4146-9893-4c1570c436da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:59.103666 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794d7aa-ea5d-4146-9893-4c1570c436da None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:59.104036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794d7aa-ea5d-4146-9893-4c1570c436da None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:59.104307 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794d7aa-ea5d-4146-9893-4c1570c436da None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:59.105076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794d7aa-ea5d-4146-9893-4c1570c436da None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:59.105378 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794d7aa-ea5d-4146-9893-4c1570c436da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:59.106893 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794d7aa-ea5d-4146-9893-4c1570c436da None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:59.135369 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 333/663] 10.222.0.76 () {56 vars in 1137 bytes} [Wed Aug 24 15:13:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:59.137717 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794d7aa-ea5d-4146-9893-4c1570c436da None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:59.162462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3876ea-121f-4f7e-8e5d-2099ff722cdb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:59.221224 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51c273-abda-4f36-a4d5-7f15a002a07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:59.222910 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51c273-abda-4f36-a4d5-7f15a002a07f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:59.223433 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51c273-abda-4f36-a4d5-7f15a002a07f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:59.223753 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51c273-abda-4f36-a4d5-7f15a002a07f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:59.236208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51c273-abda-4f36-a4d5-7f15a002a07f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:13:59.242930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ea51c273-abda-4f36-a4d5-7f15a002a07f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:59.242930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:59.242930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:59.242930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:59.242930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:59.242930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:59.242930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:59.242930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:59.242930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:59.242930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:59.242930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:59.242930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:59.242930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:13:59.242930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:13:59.242930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:13:59.242930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:13:59.244830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:13:59.244830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:13:59.244830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:13:59.244830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:13:59.244830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:59.244830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:59.244830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:59.244830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:59.244830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:59.244830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:59.244830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:59.244830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:59.244830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:59.244830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:59.244830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:59.244830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:59.245705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:59.245705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:59.245705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:59.245705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:59.245705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:13:59.245705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:13:59.245705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:13:59.245705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:13:59.245705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:13:59.245705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:13:59.245705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:13:59.245705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:13:59.245705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 331/664] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:13:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:59.245705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51c273-abda-4f36-a4d5-7f15a002a07f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:59.283025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794d7aa-ea5d-4146-9893-4c1570c436da None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:59.303436 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:59.307992 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:59.335580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1a863-3fb4-439d-b902-cb60c03ca2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:59.336390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1a863-3fb4-439d-b902-cb60c03ca2b9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:59.336718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1a863-3fb4-439d-b902-cb60c03ca2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:59.337044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a1a863-3fb4-439d-b902-cb60c03ca2b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:59.345951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a1a863-3fb4-439d-b902-cb60c03ca2b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:59.347451 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a1a863-3fb4-439d-b902-cb60c03ca2b9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:59.360213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-f4a1a863-3fb4-439d-b902-cb60c03ca2b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:59.360213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:13:59.360213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:13:59.360213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:13:59.360213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:13:59.360213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:13:59.360213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:13:59.360213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:13:59.360213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:13:59.360213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:13:59.360213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:13:59.360213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:13:59.360213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:13:59.360213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:13:59.360213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:13:59.361177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:13:59.362066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:13:59.362066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:13:59.362066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:13:59.362066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:13:59.362066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:13:59.362066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:13:59.362066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:13:59.362066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:13:59.362066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 334/665] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:13:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:13:59.362594 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1a863-3fb4-439d-b902-cb60c03ca2b9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:59.387400 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51c273-abda-4f36-a4d5-7f15a002a07f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:59.435696 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d4132-118d-4d82-94a1-d41f5c1141fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:59.436414 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d4132-118d-4d82-94a1-d41f5c1141fa None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:59.436799 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d4132-118d-4d82-94a1-d41f5c1141fa None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:59.437172 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15d4132-118d-4d82-94a1-d41f5c1141fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:59.510865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a1a863-3fb4-439d-b902-cb60c03ca2b9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:59.551679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d4132-118d-4d82-94a1-d41f5c1141fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:59.553148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15d4132-118d-4d82-94a1-d41f5c1141fa None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:59.559783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e79c54-9b90-488d-a8a3-88df55aa1db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:59.560389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e79c54-9b90-488d-a8a3-88df55aa1db6 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:59.560668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e79c54-9b90-488d-a8a3-88df55aa1db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:59.561490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e79c54-9b90-488d-a8a3-88df55aa1db6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:59.604399 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:59.612131 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:59.641917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e79c54-9b90-488d-a8a3-88df55aa1db6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:59.643289 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e79c54-9b90-488d-a8a3-88df55aa1db6 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:59.686188 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 332/666] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:59.686188 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d4132-118d-4d82-94a1-d41f5c1141fa None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:59.769749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 335/667] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:13:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:13:59.771352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e79c54-9b90-488d-a8a3-88df55aa1db6 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:59.801123 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15d4132-118d-4d82-94a1-d41f5c1141fa None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:59.844202 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0ebb1-91e6-4db7-a881-e2060b101e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:59.845049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0ebb1-91e6-4db7-a881-e2060b101e74 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:59.845243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0ebb1-91e6-4db7-a881-e2060b101e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:59.845472 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b0ebb1-91e6-4db7-a881-e2060b101e74 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:59.846102 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0ebb1-91e6-4db7-a881-e2060b101e74 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:59.846514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0ebb1-91e6-4db7-a881-e2060b101e74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:59.847826 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b0ebb1-91e6-4db7-a881-e2060b101e74 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:59.854448 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:59.861949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:13:59.862948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 333/668] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:13:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:59.863873 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0ebb1-91e6-4db7-a881-e2060b101e74 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:59.890122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e79c54-9b90-488d-a8a3-88df55aa1db6 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:13:59.929326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ca561-1481-42f5-9f68-6738b4ef387e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:13:59.929934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ca561-1481-42f5-9f68-6738b4ef387e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:13:59.930211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ca561-1481-42f5-9f68-6738b4ef387e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:13:59.930510 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ca561-1481-42f5-9f68-6738b4ef387e None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:13:59.931195 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ca561-1481-42f5-9f68-6738b4ef387e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:13:59.931499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ca561-1481-42f5-9f68-6738b4ef387e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:13:59.932920 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9ca561-1481-42f5-9f68-6738b4ef387e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:13:59.946340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 336/669] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:13:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:13:59.951420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ca561-1481-42f5-9f68-6738b4ef387e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:13:59.973101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b0ebb1-91e6-4db7-a881-e2060b101e74 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:00.009329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebc4af-704d-4e92-9bb6-0b987080810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:00.009952 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebc4af-704d-4e92-9bb6-0b987080810a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:00.010366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebc4af-704d-4e92-9bb6-0b987080810a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:00.010639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ebc4af-704d-4e92-9bb6-0b987080810a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:00.022284 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ebc4af-704d-4e92-9bb6-0b987080810a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:00.023774 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ebc4af-704d-4e92-9bb6-0b987080810a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:00.040461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-31ebc4af-704d-4e92-9bb6-0b987080810a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:00.040461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:00.040461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:00.040461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:00.040461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:00.040461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:00.040461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:00.040461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:00.040461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:00.040461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:00.040461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:00.040461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:00.040461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:00.040461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:00.040461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:00.040461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:00.041409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:00.042327 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:00.042327 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:00.042327 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:00.042327 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 334/670] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:00.042563 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebc4af-704d-4e92-9bb6-0b987080810a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:00.082213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ca561-1481-42f5-9f68-6738b4ef387e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:00.106140 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:00.119888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff5a7f-1676-449c-9257-cd2b005db8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:00.120516 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff5a7f-1676-449c-9257-cd2b005db8e9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:00.120805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff5a7f-1676-449c-9257-cd2b005db8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:00.121074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff5a7f-1676-449c-9257-cd2b005db8e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:00.123167 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:00.133477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff5a7f-1676-449c-9257-cd2b005db8e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:00.134979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff5a7f-1676-449c-9257-cd2b005db8e9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:00.149540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-27ff5a7f-1676-449c-9257-cd2b005db8e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:00.149540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:00.149540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:00.149540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:00.149540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:00.149540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:00.149540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:00.149540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:00.149540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:00.149540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:00.149540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:00.149540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:00.149540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:00.149540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:00.149540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:00.149540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:00.150560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:00.151753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:00.151753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:00.151753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:00.154549 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 337/671] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:13:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:00.156155 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff5a7f-1676-449c-9257-cd2b005db8e9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:00.164846 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ebc4af-704d-4e92-9bb6-0b987080810a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:00.202310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b604c-9aba-4193-8860-dfb20823b333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:00.203015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b604c-9aba-4193-8860-dfb20823b333 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:00.203375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b604c-9aba-4193-8860-dfb20823b333 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:00.203700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8b604c-9aba-4193-8860-dfb20823b333 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:00.218073 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b604c-9aba-4193-8860-dfb20823b333 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:00.219959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8b604c-9aba-4193-8860-dfb20823b333 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:00.235296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-9e8b604c-9aba-4193-8860-dfb20823b333 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:00.235296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:00.235296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:00.235296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:00.235296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:00.235296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:00.235296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:00.235296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:00.235296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:00.235296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:00.235296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:00.235296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:00.235296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:00.235296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:00.235296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:00.235296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:00.236936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:00.237893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:00.237893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:00.237893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:00.239804 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 335/672] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:00.240734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b604c-9aba-4193-8860-dfb20823b333 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:00.295535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff5a7f-1676-449c-9257-cd2b005db8e9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:00.333111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178487f-200a-4de8-a848-d2187f227e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:00.333712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178487f-200a-4de8-a848-d2187f227e9c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:00.333980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178487f-200a-4de8-a848-d2187f227e9c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:00.334269 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8178487f-200a-4de8-a848-d2187f227e9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:00.334991 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178487f-200a-4de8-a848-d2187f227e9c None admin] RBAC: Adding query filter params (role.id=59a2b50693a646fc8332b835233486ce, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=8a5fd31a2db944f596912e4f0aa26835) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:00.335289 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178487f-200a-4de8-a848-d2187f227e9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:00.336703 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8178487f-200a-4de8-a848-d2187f227e9c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:00.354752 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 338/673] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:14:00 2022] GET /identity/v3/role_assignments?user.id=8a5fd31a2db944f596912e4f0aa26835&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=59a2b50693a646fc8332b835233486ce => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:00.356444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178487f-200a-4de8-a848-d2187f227e9c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:00.359238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8b604c-9aba-4193-8860-dfb20823b333 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:00.363908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:00.409950 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cec825-92f4-4da1-b368-58931b1fcb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:00.413456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cec825-92f4-4da1-b368-58931b1fcb3a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:00.413915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cec825-92f4-4da1-b368-58931b1fcb3a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:00.415342 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cec825-92f4-4da1-b368-58931b1fcb3a None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:00.415342 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cec825-92f4-4da1-b368-58931b1fcb3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:00.415342 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cec825-92f4-4da1-b368-58931b1fcb3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:00.417148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cec825-92f4-4da1-b368-58931b1fcb3a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:00.432545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 336/674] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:00.433554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cec825-92f4-4da1-b368-58931b1fcb3a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:00.437354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:00.505961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8178487f-200a-4de8-a848-d2187f227e9c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:00.552488 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf47c3-ed7d-43af-ac25-7d78e7f808ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:00.553266 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf47c3-ed7d-43af-ac25-7d78e7f808ce None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:00.553539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf47c3-ed7d-43af-ac25-7d78e7f808ce None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:00.554210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecf47c3-ed7d-43af-ac25-7d78e7f808ce None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:00.554970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf47c3-ed7d-43af-ac25-7d78e7f808ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:00.555192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf47c3-ed7d-43af-ac25-7d78e7f808ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:00.556560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecf47c3-ed7d-43af-ac25-7d78e7f808ce None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:00.569276 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 339/675] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:00.570126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf47c3-ed7d-43af-ac25-7d78e7f808ce None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:00.575318 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cec825-92f4-4da1-b368-58931b1fcb3a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:00.619451 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd605d05-abae-4d36-8c26-5e1442f8b3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:00.620118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd605d05-abae-4d36-8c26-5e1442f8b3b0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:00.620397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd605d05-abae-4d36-8c26-5e1442f8b3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:00.620667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd605d05-abae-4d36-8c26-5e1442f8b3b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:00.621316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd605d05-abae-4d36-8c26-5e1442f8b3b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:00.621654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd605d05-abae-4d36-8c26-5e1442f8b3b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:00.623736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd605d05-abae-4d36-8c26-5e1442f8b3b0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:00.637611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 337/676] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:00.639058 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd605d05-abae-4d36-8c26-5e1442f8b3b0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:00.667661 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:00.711155 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecf47c3-ed7d-43af-ac25-7d78e7f808ce None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:00.730041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:00.763708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dc06b-1476-47d4-896f-fa01efe5d5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:00.764353 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dc06b-1476-47d4-896f-fa01efe5d5fb None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:00.764696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dc06b-1476-47d4-896f-fa01efe5d5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:00.765030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12dc06b-1476-47d4-896f-fa01efe5d5fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:00.774164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dc06b-1476-47d4-896f-fa01efe5d5fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:00.775723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12dc06b-1476-47d4-896f-fa01efe5d5fb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:00.787410 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd605d05-abae-4d36-8c26-5e1442f8b3b0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:00.788899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-b12dc06b-1476-47d4-896f-fa01efe5d5fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:00.788899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:00.788899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:00.788899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:00.788899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:00.788899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:00.788899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:00.788899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:00.788899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:00.788899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:00.788899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:00.788899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:00.788899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:00.788899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:00.788899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:00.789960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:00.791216 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:00.791216 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:00.791216 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:00.791216 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:00.791216 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:00.791216 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:00.791216 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:00.791216 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:00.796514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 340/677] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:00.797426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dc06b-1476-47d4-896f-fa01efe5d5fb None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:00.842445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32eaa3-b3e7-47eb-9150-e032a3ec6447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:00.843095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32eaa3-b3e7-47eb-9150-e032a3ec6447 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:00.843372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32eaa3-b3e7-47eb-9150-e032a3ec6447 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:00.843713 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c32eaa3-b3e7-47eb-9150-e032a3ec6447 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:00.854938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c32eaa3-b3e7-47eb-9150-e032a3ec6447 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:00.856795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c32eaa3-b3e7-47eb-9150-e032a3ec6447 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:00.872673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7c32eaa3-b3e7-47eb-9150-e032a3ec6447 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:00.872673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:00.872673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:00.872673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:00.872673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:00.872673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:00.872673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:00.872673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:00.872673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:00.872673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:00.872673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:00.872673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:00.872673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:00.872673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:00.872673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:00.874529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:00.876156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:00.876156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:00.876156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:00.876156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:00.876156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:00.876156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:00.876156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:00.876156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:00.878520 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 338/678] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:00.879616 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32eaa3-b3e7-47eb-9150-e032a3ec6447 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:00.930108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12dc06b-1476-47d4-896f-fa01efe5d5fb None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:00.981001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72262329-5c8d-49ee-b90b-3a33f07018f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:00.982249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72262329-5c8d-49ee-b90b-3a33f07018f1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:00.982731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72262329-5c8d-49ee-b90b-3a33f07018f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:00.983153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72262329-5c8d-49ee-b90b-3a33f07018f1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:00.992492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72262329-5c8d-49ee-b90b-3a33f07018f1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:00.994104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72262329-5c8d-49ee-b90b-3a33f07018f1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:01.006653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-72262329-5c8d-49ee-b90b-3a33f07018f1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:14:01.006653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:01.006653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:01.006653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:01.006653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:01.006653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:01.006653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:01.006653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:01.006653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:01.006653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:01.006653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:01.006653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:01.006653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:01.006653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:01.006653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:01.006653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:01.008489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:01.010667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:01.010667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:01.010667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:01.010667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:01.010667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:01.010667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:01.010667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:01.010667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:01.010667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:14:01.010667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:01.010667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 341/679] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:14:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:01.012361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72262329-5c8d-49ee-b90b-3a33f07018f1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:01.021268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:01.046202 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c32eaa3-b3e7-47eb-9150-e032a3ec6447 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:01.047463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:01.085864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d0f2f-6310-441c-9a9a-289fe3394ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:01.086671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d0f2f-6310-441c-9a9a-289fe3394ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:01.087044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d0f2f-6310-441c-9a9a-289fe3394ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:01.087787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d0f2f-6310-441c-9a9a-289fe3394ba4 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:01.088299 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d0f2f-6310-441c-9a9a-289fe3394ba4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:01.088716 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d0f2f-6310-441c-9a9a-289fe3394ba4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:01.090397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d0f2f-6310-441c-9a9a-289fe3394ba4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:01.105193 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 339/680] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:01.106701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d0f2f-6310-441c-9a9a-289fe3394ba4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:01.137058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72262329-5c8d-49ee-b90b-3a33f07018f1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:01.174071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1284fde0-1bfe-493b-a791-c918def9c39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:01.174972 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284fde0-1bfe-493b-a791-c918def9c39a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:01.175252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284fde0-1bfe-493b-a791-c918def9c39a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:01.175643 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1284fde0-1bfe-493b-a791-c918def9c39a None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:01.176342 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1284fde0-1bfe-493b-a791-c918def9c39a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:01.176674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1284fde0-1bfe-493b-a791-c918def9c39a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:01.178116 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1284fde0-1bfe-493b-a791-c918def9c39a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:01.191530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 342/681] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:01.193286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dbebc-dfbb-48d5-aa37-2668128485fb None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:01.193573 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dbebc-dfbb-48d5-aa37-2668128485fb None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:01.193846 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524dbebc-dfbb-48d5-aa37-2668128485fb None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:01.195269 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 343/682] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:01.200050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dbebc-dfbb-48d5-aa37-2668128485fb None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:01.221792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d0f2f-6310-441c-9a9a-289fe3394ba4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:01.259957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a8f06-05fc-4d3f-94d7-331834b39b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:01.260628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a8f06-05fc-4d3f-94d7-331834b39b7c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:01.260858 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a8f06-05fc-4d3f-94d7-331834b39b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:01.261106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a8f06-05fc-4d3f-94d7-331834b39b7c None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:01.261833 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a8f06-05fc-4d3f-94d7-331834b39b7c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:01.262133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a8f06-05fc-4d3f-94d7-331834b39b7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:01.263567 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a8f06-05fc-4d3f-94d7-331834b39b7c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:01.289943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 340/683] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 24 15:14:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:01.290944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a8f06-05fc-4d3f-94d7-331834b39b7c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:01.294399 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:01.311341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:01.323382 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524dbebc-dfbb-48d5-aa37-2668128485fb None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:01.361354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad1d760-e653-4d29-b743-370a8a456853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:01.361965 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1d760-e653-4d29-b743-370a8a456853 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:01.362265 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1d760-e653-4d29-b743-370a8a456853 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:01.362556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad1d760-e653-4d29-b743-370a8a456853 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:01.363374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1d760-e653-4d29-b743-370a8a456853 None admin] RBAC: Adding query filter params (role.id=59a2b50693a646fc8332b835233486ce, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=5427bd0ac88042338b956c10cfd07baa) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:01.363588 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1d760-e653-4d29-b743-370a8a456853 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:01.365049 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad1d760-e653-4d29-b743-370a8a456853 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:01.374372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 344/684] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:14:01 2022] GET /identity/v3/role_assignments?user.id=5427bd0ac88042338b956c10cfd07baa&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=59a2b50693a646fc8332b835233486ce => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:01.375783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d18628-c9f8-4e07-9fd7-39a1e5e4d4f8 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:01.376135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d18628-c9f8-4e07-9fd7-39a1e5e4d4f8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:01.376475 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d18628-c9f8-4e07-9fd7-39a1e5e4d4f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:01.397787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-26d18628-c9f8-4e07-9fd7-39a1e5e4d4f8 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:01.399161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-26d18628-c9f8-4e07-9fd7-39a1e5e4d4f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:01.419984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a8f06-05fc-4d3f-94d7-331834b39b7c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:01.473979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583e0a3-f316-491d-a140-5991e49e18ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:01.474733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583e0a3-f316-491d-a140-5991e49e18ed None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:01.475018 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583e0a3-f316-491d-a140-5991e49e18ed None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:01.475292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583e0a3-f316-491d-a140-5991e49e18ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:01.476132 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583e0a3-f316-491d-a140-5991e49e18ed None admin] RBAC: Adding query filter params (role.id=59a2b50693a646fc8332b835233486ce, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=194c7492cf954a6a8dd21de66aa24fec) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:01.476447 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583e0a3-f316-491d-a140-5991e49e18ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:01.477992 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583e0a3-f316-491d-a140-5991e49e18ed None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:01.483349 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 345/685] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:14:01.484289 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d18628-c9f8-4e07-9fd7-39a1e5e4d4f8 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:01.494048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 341/686] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:14:01 2022] GET /identity/v3/role_assignments?user.id=194c7492cf954a6a8dd21de66aa24fec&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=59a2b50693a646fc8332b835233486ce => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:01.602902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:01.613226 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d18628-c9f8-4e07-9fd7-39a1e5e4d4f8 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:01.653796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dc70c-28c2-4975-a510-2a252b6cc3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:01.654462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dc70c-28c2-4975-a510-2a252b6cc3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:01.655120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dc70c-28c2-4975-a510-2a252b6cc3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:01.655459 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dc70c-28c2-4975-a510-2a252b6cc3d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:01.751695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00dc70c-28c2-4975-a510-2a252b6cc3d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:01.754371 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00dc70c-28c2-4975-a510-2a252b6cc3d4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:01.856373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583e0a3-f316-491d-a140-5991e49e18ed None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:01.886074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:01.909869 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 346/687] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:14:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:14:01.915456 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dc70c-28c2-4975-a510-2a252b6cc3d4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:01.935792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:01.981224 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583e0a3-f316-491d-a140-5991e49e18ed None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:02.029994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cca348-7309-46da-8609-733c5d2c0ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:02.030870 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cca348-7309-46da-8609-733c5d2c0ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:02.031383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cca348-7309-46da-8609-733c5d2c0ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:02.032889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cca348-7309-46da-8609-733c5d2c0ee3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:02.053074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dc70c-28c2-4975-a510-2a252b6cc3d4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:02.062970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cca348-7309-46da-8609-733c5d2c0ee3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:02.064521 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cca348-7309-46da-8609-733c5d2c0ee3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:02.093532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d4cca348-7309-46da-8609-733c5d2c0ee3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:02.093532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:02.093532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:02.093532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:02.093532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:02.093532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:02.093532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:02.093532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:02.093532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:02.093532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:02.093532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:02.093532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:02.093532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:02.093532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:02.093532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:02.093532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:02.094685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:02.095703 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:02.095703 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:02.095703 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:02.095703 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 342/688] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:02.104790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cca348-7309-46da-8609-733c5d2c0ee3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:02.130746 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352b68b-fedf-47ab-9da2-6d63b06a711e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:02.131790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352b68b-fedf-47ab-9da2-6d63b06a711e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:02.132211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352b68b-fedf-47ab-9da2-6d63b06a711e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:02.132604 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4352b68b-fedf-47ab-9da2-6d63b06a711e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:02.157148 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4352b68b-fedf-47ab-9da2-6d63b06a711e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:02.158628 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4352b68b-fedf-47ab-9da2-6d63b06a711e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:02.190765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-4352b68b-fedf-47ab-9da2-6d63b06a711e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:02.190765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:02.190765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:02.190765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:02.190765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:02.190765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:02.190765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:02.190765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:02.190765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:02.190765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:02.190765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:02.190765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:02.190765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:02.190765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:02.190765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:02.190765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:02.191780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:02.193346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:02.193346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:02.193346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:02.193346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 347/689] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:02.199488 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352b68b-fedf-47ab-9da2-6d63b06a711e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:02.252170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:02.326580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cca348-7309-46da-8609-733c5d2c0ee3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:02.368885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:02.375574 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4352b68b-fedf-47ab-9da2-6d63b06a711e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:02.404414 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03814988-e70a-4b2e-8b03-e40240337be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:02.405372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03814988-e70a-4b2e-8b03-e40240337be5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:02.405831 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03814988-e70a-4b2e-8b03-e40240337be5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:02.412423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03814988-e70a-4b2e-8b03-e40240337be5 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:02.413767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03814988-e70a-4b2e-8b03-e40240337be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:02.414301 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03814988-e70a-4b2e-8b03-e40240337be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:02.416326 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03814988-e70a-4b2e-8b03-e40240337be5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:02.426794 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe495d-e511-448d-aa29-5e072db4e16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:02.427428 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe495d-e511-448d-aa29-5e072db4e16b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:02.427712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe495d-e511-448d-aa29-5e072db4e16b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:02.428189 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe495d-e511-448d-aa29-5e072db4e16b None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:02.428979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe495d-e511-448d-aa29-5e072db4e16b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:02.429283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe495d-e511-448d-aa29-5e072db4e16b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:02.430719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe495d-e511-448d-aa29-5e072db4e16b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:02.442250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 343/690] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:02.447759 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 348/691] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:02.449611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03814988-e70a-4b2e-8b03-e40240337be5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:02.462741 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe495d-e511-448d-aa29-5e072db4e16b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:02.642583 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:02.659591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe495d-e511-448d-aa29-5e072db4e16b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:02.681298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03814988-e70a-4b2e-8b03-e40240337be5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:02.720910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46feffb-8d0f-4b26-a741-942444a7e8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:02.722043 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46feffb-8d0f-4b26-a741-942444a7e8dc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:02.722351 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46feffb-8d0f-4b26-a741-942444a7e8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:02.722624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46feffb-8d0f-4b26-a741-942444a7e8dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:02.732960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46feffb-8d0f-4b26-a741-942444a7e8dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:02.734445 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46feffb-8d0f-4b26-a741-942444a7e8dc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:02.734581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91737f1a-bf75-4f60-b385-6fe0635dab8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:02.735378 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91737f1a-bf75-4f60-b385-6fe0635dab8a None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:02.735654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91737f1a-bf75-4f60-b385-6fe0635dab8a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:02.735923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91737f1a-bf75-4f60-b385-6fe0635dab8a None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:02.736701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91737f1a-bf75-4f60-b385-6fe0635dab8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:02.738325 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91737f1a-bf75-4f60-b385-6fe0635dab8a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:02.754591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-e46feffb-8d0f-4b26-a741-942444a7e8dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:02.754591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:02.754591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:02.754591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:02.754591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:02.754591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:02.754591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:02.754591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:02.754591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:02.754591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:02.754591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:02.754591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:02.754591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:02.754591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:02.754591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:02.755985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:02.756888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:02.756888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:02.756888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:02.756888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:02.756888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:02.756888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:02.756888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:02.756888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:02.757328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 349/692] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:02.762249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52efeb-9ee7-4aad-bc60-4c841ff92e09 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:02.762249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52efeb-9ee7-4aad-bc60-4c841ff92e09 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:02.762249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f52efeb-9ee7-4aad-bc60-4c841ff92e09 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:02.766478 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 350/693] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:02.768065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b1299-a1a5-4f9d-b544-d535d994b74f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:02.768348 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b1299-a1a5-4f9d-b544-d535d994b74f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:02.768614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2b1299-a1a5-4f9d-b544-d535d994b74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:02.773319 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 344/694] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:14:02 2022] POST /identity/v3/users => generated 254 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:14:02.774829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91737f1a-bf75-4f60-b385-6fe0635dab8a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:02.825019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:02.835043 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0c2b1299-a1a5-4f9d-b544-d535d994b74f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:02.838147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0c2b1299-a1a5-4f9d-b544-d535d994b74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:02.931610 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91737f1a-bf75-4f60-b385-6fe0635dab8a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:02.976938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 351/695] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:14:02.985870 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03757911-c548-40e5-a254-b93edfc5d587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:02.986536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03757911-c548-40e5-a254-b93edfc5d587 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:02.986818 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03757911-c548-40e5-a254-b93edfc5d587 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:02.987091 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03757911-c548-40e5-a254-b93edfc5d587 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:02.987978 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03757911-c548-40e5-a254-b93edfc5d587 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:02.988310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03757911-c548-40e5-a254-b93edfc5d587 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:02.989936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03757911-c548-40e5-a254-b93edfc5d587 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:03.010845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 345/696] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:03.017092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b1299-a1a5-4f9d-b544-d535d994b74f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:03.136477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:03.157121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2b1299-a1a5-4f9d-b544-d535d994b74f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:03.198899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc880e1-d27a-49a9-81d2-6fac12a79038 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:03.199678 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc880e1-d27a-49a9-81d2-6fac12a79038 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:03.200100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc880e1-d27a-49a9-81d2-6fac12a79038 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:03.200370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc880e1-d27a-49a9-81d2-6fac12a79038 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:03.201106 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc880e1-d27a-49a9-81d2-6fac12a79038 None admin] RBAC: Adding query filter params (role.id=59a2b50693a646fc8332b835233486ce, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=0679072b37db41e880a95b738bea47ea) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:03.201409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc880e1-d27a-49a9-81d2-6fac12a79038 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:03.202913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc880e1-d27a-49a9-81d2-6fac12a79038 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:03.210531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 352/697] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:14:03 2022] GET /identity/v3/role_assignments?user.id=0679072b37db41e880a95b738bea47ea&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=59a2b50693a646fc8332b835233486ce => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:03.397813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03757911-c548-40e5-a254-b93edfc5d587 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:03.511428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:03.519630 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03757911-c548-40e5-a254-b93edfc5d587 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:03.557679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab0ca9-11c4-452d-a09b-e3b6d455506e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:03.558338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab0ca9-11c4-452d-a09b-e3b6d455506e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:03.558615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab0ca9-11c4-452d-a09b-e3b6d455506e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:03.558879 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab0ca9-11c4-452d-a09b-e3b6d455506e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:03.570105 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab0ca9-11c4-452d-a09b-e3b6d455506e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:03.574991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-18ab0ca9-11c4-452d-a09b-e3b6d455506e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:03.574991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:03.574991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:03.574991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:03.574991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:03.574991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:03.574991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:03.574991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:03.574991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:03.574991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:03.574991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:03.574991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:03.574991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:03.574991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:03.574991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:03.574991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:03.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:03.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:03.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:03.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:03.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:03.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:03.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:03.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:03.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:03.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:03.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:03.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:03.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:03.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:03.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:03.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:03.576744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:03.576744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:03.576744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:03.576744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:03.576744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:03.576744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:03.576744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:03.576744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:03.576744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:03.576744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:03.576744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:03.576744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:03.576744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 346/698] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:14:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:03.583870 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc880e1-d27a-49a9-81d2-6fac12a79038 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:03.713096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc880e1-d27a-49a9-81d2-6fac12a79038 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:03.751145 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede8908-55d1-408b-9b87-8a6763f1c954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:03.752112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede8908-55d1-408b-9b87-8a6763f1c954 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:03.752561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede8908-55d1-408b-9b87-8a6763f1c954 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:03.753037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede8908-55d1-408b-9b87-8a6763f1c954 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:03.754060 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede8908-55d1-408b-9b87-8a6763f1c954 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:03.754677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede8908-55d1-408b-9b87-8a6763f1c954 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:03.757559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede8908-55d1-408b-9b87-8a6763f1c954 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:03.767764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 353/699] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:14:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:03.774149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab0ca9-11c4-452d-a09b-e3b6d455506e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:03.889211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab0ca9-11c4-452d-a09b-e3b6d455506e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:03.926328 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88a04d-319b-4dc8-b0be-5514d885f51e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:03.927806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88a04d-319b-4dc8-b0be-5514d885f51e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:03.928088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88a04d-319b-4dc8-b0be-5514d885f51e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:03.928353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88a04d-319b-4dc8-b0be-5514d885f51e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:03.938890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88a04d-319b-4dc8-b0be-5514d885f51e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:03.940326 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88a04d-319b-4dc8-b0be-5514d885f51e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:03.956737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1c88a04d-319b-4dc8-b0be-5514d885f51e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:03.956737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:03.956737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:03.956737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:03.956737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:03.956737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:03.956737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:03.956737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:03.956737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:03.956737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:03.956737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:03.956737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:03.956737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:03.956737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:03.956737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:03.956737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:03.957750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:03.958721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:03.958721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:03.958721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:03.958721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 347/700] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:03.964608 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede8908-55d1-408b-9b87-8a6763f1c954 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:03.996334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:04.118102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede8908-55d1-408b-9b87-8a6763f1c954 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:04.150196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b73941-aeaa-491e-b2a4-5730e697a1f0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:04.150664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b73941-aeaa-491e-b2a4-5730e697a1f0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:04.151077 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b73941-aeaa-491e-b2a4-5730e697a1f0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:04.153913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 348/701] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:04.158399 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c775e63-c055-4281-876b-4af68b9ee311 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:04.158755 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c775e63-c055-4281-876b-4af68b9ee311 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:04.158944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c775e63-c055-4281-876b-4af68b9ee311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:04.162748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d1def-2c72-4ab6-a7d2-b6e235a179e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:04.163421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d1def-2c72-4ab6-a7d2-b6e235a179e3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:04.163703 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d1def-2c72-4ab6-a7d2-b6e235a179e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:04.163972 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d1def-2c72-4ab6-a7d2-b6e235a179e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:04.164627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d1def-2c72-4ab6-a7d2-b6e235a179e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:04.164933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d1def-2c72-4ab6-a7d2-b6e235a179e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:04.166326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d1def-2c72-4ab6-a7d2-b6e235a179e3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:04.177933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:04.182923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0c775e63-c055-4281-876b-4af68b9ee311 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:04.185791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0c775e63-c055-4281-876b-4af68b9ee311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:04.191871 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 354/702] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:04.196912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d1def-2c72-4ab6-a7d2-b6e235a179e3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:04.278357 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 349/703] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:14:04.279849 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fc518-54ce-41bb-a80c-73d4a8f5356f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:04.280130 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fc518-54ce-41bb-a80c-73d4a8f5356f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:04.280396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28fc518-54ce-41bb-a80c-73d4a8f5356f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:04.281986 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 350/704] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:04.289877 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005becc-148c-4894-afaf-6a0d157a585a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:04.290157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005becc-148c-4894-afaf-6a0d157a585a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:04.290447 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2005becc-148c-4894-afaf-6a0d157a585a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:04.292513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:04.327702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d1def-2c72-4ab6-a7d2-b6e235a179e3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:04.336117 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2005becc-148c-4894-afaf-6a0d157a585a None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:04.337370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2005becc-148c-4894-afaf-6a0d157a585a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:04.380828 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62426c-b74d-49f3-91f1-624b07b2de85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:04.381393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62426c-b74d-49f3-91f1-624b07b2de85 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:04.381666 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62426c-b74d-49f3-91f1-624b07b2de85 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:04.381929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb62426c-b74d-49f3-91f1-624b07b2de85 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:04.391036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb62426c-b74d-49f3-91f1-624b07b2de85 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:04.392639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb62426c-b74d-49f3-91f1-624b07b2de85 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:04.406370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-cb62426c-b74d-49f3-91f1-624b07b2de85 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:14:04.406370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:04.406370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:04.406370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:04.406370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:04.406370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:04.406370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:04.406370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:04.406370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:04.406370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:04.406370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:04.406370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:04.406370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:04.406370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:04.406370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:04.406370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:04.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:04.408341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:04.408341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:04.408341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:04.408341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:04.408341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:04.408341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:04.408341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:04.408341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:04.408341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:14:04.408341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:04.408341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 355/705] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 15:14:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:04.413494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62426c-b74d-49f3-91f1-624b07b2de85 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:04.515948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 351/706] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:14:04.541226 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb62426c-b74d-49f3-91f1-624b07b2de85 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:04.574050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:04.580770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f62da-340c-4181-9236-a2b546fe06ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:04.581611 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f62da-340c-4181-9236-a2b546fe06ee None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:04.582124 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f62da-340c-4181-9236-a2b546fe06ee None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:04.582566 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370f62da-340c-4181-9236-a2b546fe06ee None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:04.583465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f62da-340c-4181-9236-a2b546fe06ee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:04.583929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f62da-340c-4181-9236-a2b546fe06ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:04.585559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370f62da-340c-4181-9236-a2b546fe06ee None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:04.604651 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 356/707] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 15:14:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:04.609651 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005becc-148c-4894-afaf-6a0d157a585a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:04.610901 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f62da-340c-4181-9236-a2b546fe06ee None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:04.741147 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:04.813389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370f62da-340c-4181-9236-a2b546fe06ee None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:04.819948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2005becc-148c-4894-afaf-6a0d157a585a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:04.874656 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8450f-079b-40cf-ba63-2aec1763f0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:04.875558 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8450f-079b-40cf-ba63-2aec1763f0a0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:04.875983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8450f-079b-40cf-ba63-2aec1763f0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:04.876401 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d8450f-079b-40cf-ba63-2aec1763f0a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:04.901877 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef704c3-008f-449a-8b15-2159cf51da0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:04.903016 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef704c3-008f-449a-8b15-2159cf51da0d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:04.903144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef704c3-008f-449a-8b15-2159cf51da0d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:04.903454 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef704c3-008f-449a-8b15-2159cf51da0d None admin] PATH_INFO: `/v3/services/network` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:04.904455 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef704c3-008f-449a-8b15-2159cf51da0d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:04.906458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef704c3-008f-449a-8b15-2159cf51da0d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-aef704c3-008f-449a-8b15-2159cf51da0d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:04.920126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:04.921538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:04.921538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:04.921538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:04.921538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:04.921538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:04.921538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:04.921538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:04.921538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:04.921538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:04.921538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:04.921538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:04.921538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:04.921538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:04.921538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:04.921538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:04.921538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:04.922832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:04.922832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:04.922832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:04.922832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:04.922832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:04.922832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:04.922832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:04.922832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:04.922832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:04.922832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:04.922832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:04.922832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:04.922832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:14:04.922832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:04.928145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 352/708] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:04 2022] GET /identity/v3/services/network => generated 88 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:04.929402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef704c3-008f-449a-8b15-2159cf51da0d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:04.951771 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:04.972938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8450f-079b-40cf-ba63-2aec1763f0a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:04.974810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d8450f-079b-40cf-ba63-2aec1763f0a0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:05.057286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef704c3-008f-449a-8b15-2159cf51da0d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:05.094576 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:05.097706 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 357/709] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:14:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:14:05.098682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8450f-079b-40cf-ba63-2aec1763f0a0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:05.098834 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16401eb0-9939-41b2-89a0-b89f75174cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:05.099500 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16401eb0-9939-41b2-89a0-b89f75174cac None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:05.099783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16401eb0-9939-41b2-89a0-b89f75174cac None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:05.100060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16401eb0-9939-41b2-89a0-b89f75174cac None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:05.100654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16401eb0-9939-41b2-89a0-b89f75174cac None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:05.102498 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16401eb0-9939-41b2-89a0-b89f75174cac None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-16401eb0-9939-41b2-89a0-b89f75174cac None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:05.112571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:05.113618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:05.113618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:05.113618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:05.113618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:05.113618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:05.113618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:05.113618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:05.113618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:05.113618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:05.113618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:05.113618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:05.113618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:05.113618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:05.113618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:05.113618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:05.113618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:05.114613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:05.114613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:05.114613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:05.114613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:05.114613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:05.114613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:05.114613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:05.114613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:05.114613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:05.114613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:05.114613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:05.114613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:05.114613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:14:05.114613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:05.114613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 353/710] 10.222.0.76 () {56 vars in 1129 bytes} [Wed Aug 24 15:14:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:05.119431 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16401eb0-9939-41b2-89a0-b89f75174cac None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:05.196645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:05.206863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d8450f-079b-40cf-ba63-2aec1763f0a0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:05.246719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa10a3-9901-4e11-aea4-d4856ab431f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:05.247327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa10a3-9901-4e11-aea4-d4856ab431f0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:05.247603 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa10a3-9901-4e11-aea4-d4856ab431f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:05.247871 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa10a3-9901-4e11-aea4-d4856ab431f0 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:05.248499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa10a3-9901-4e11-aea4-d4856ab431f0 None admin] RBAC: Adding query filter params (name=network) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:05.248797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa10a3-9901-4e11-aea4-d4856ab431f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:05.250171 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa10a3-9901-4e11-aea4-d4856ab431f0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:05.259915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 358/711] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 15:14:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:05.260765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa10a3-9901-4e11-aea4-d4856ab431f0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:05.281660 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16401eb0-9939-41b2-89a0-b89f75174cac None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:05.331610 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef831f-202c-4a81-8075-18f409649112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:05.332167 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef831f-202c-4a81-8075-18f409649112 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:05.332443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef831f-202c-4a81-8075-18f409649112 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:05.332713 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beef831f-202c-4a81-8075-18f409649112 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:05.353933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef831f-202c-4a81-8075-18f409649112 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:05.357152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beef831f-202c-4a81-8075-18f409649112 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:05.385682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-beef831f-202c-4a81-8075-18f409649112 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:05.385682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:05.385682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:05.385682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:05.385682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:05.385682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:05.385682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:05.385682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:05.385682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:05.385682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:05.385682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:05.385682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:05.385682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:05.385682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:05.385682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:05.385682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:05.387888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:05.389468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:05.389468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:05.389468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:05.389468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 354/712] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:05.393940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef831f-202c-4a81-8075-18f409649112 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:05.400664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa10a3-9901-4e11-aea4-d4856ab431f0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:05.458730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38fb119-3d82-4980-b9a4-a9676c12327c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:05.459774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fb119-3d82-4980-b9a4-a9676c12327c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:05.460259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fb119-3d82-4980-b9a4-a9676c12327c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:05.461561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38fb119-3d82-4980-b9a4-a9676c12327c None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:05.461561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38fb119-3d82-4980-b9a4-a9676c12327c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:05.461854 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38fb119-3d82-4980-b9a4-a9676c12327c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:05.463535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38fb119-3d82-4980-b9a4-a9676c12327c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:05.464864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:05.471035 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 359/713] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 24 15:14:05 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:05.472355 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38fb119-3d82-4980-b9a4-a9676c12327c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:05.504021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:05.555092 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beef831f-202c-4a81-8075-18f409649112 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:05.597257 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae9c7b-631d-40c3-aea0-03a7e2a7a82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:05.597789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae9c7b-631d-40c3-aea0-03a7e2a7a82f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:05.598069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae9c7b-631d-40c3-aea0-03a7e2a7a82f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:05.598930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae9c7b-631d-40c3-aea0-03a7e2a7a82f None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:05.599573 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae9c7b-631d-40c3-aea0-03a7e2a7a82f None admin] RBAC: Adding query filter params (type=network) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:05.599874 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae9c7b-631d-40c3-aea0-03a7e2a7a82f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:05.601268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae9c7b-631d-40c3-aea0-03a7e2a7a82f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:05.608054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 355/714] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 15:14:05 2022] GET /identity/v3/services?type=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:05.609184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b848ba3-88cf-43d1-a24e-7fe3629a8758 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:05.609474 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b848ba3-88cf-43d1-a24e-7fe3629a8758 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:05.609748 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b848ba3-88cf-43d1-a24e-7fe3629a8758 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:05.610180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38fb119-3d82-4980-b9a4-a9676c12327c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:05.616211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 356/715] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:05.617199 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b848ba3-88cf-43d1-a24e-7fe3629a8758 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:05.668371 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5bd60-14a7-4afc-a185-a4901b0944f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:05.669344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5bd60-14a7-4afc-a185-a4901b0944f7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:05.669796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5bd60-14a7-4afc-a185-a4901b0944f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:05.670371 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5bd60-14a7-4afc-a185-a4901b0944f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:05.671225 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5bd60-14a7-4afc-a185-a4901b0944f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:05.671700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5bd60-14a7-4afc-a185-a4901b0944f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:05.673263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5bd60-14a7-4afc-a185-a4901b0944f7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:05.688339 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 360/716] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:05.690381 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df76ff3-6af0-4239-84ca-852c68f9e039 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:05.690906 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df76ff3-6af0-4239-84ca-852c68f9e039 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:05.691535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df76ff3-6af0-4239-84ca-852c68f9e039 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:05.719750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3df76ff3-6af0-4239-84ca-852c68f9e039 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:05.721173 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3df76ff3-6af0-4239-84ca-852c68f9e039 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:05.787347 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b848ba3-88cf-43d1-a24e-7fe3629a8758 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:05.808678 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:05.836891 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 361/717] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:14:05.837802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df76ff3-6af0-4239-84ca-852c68f9e039 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:05.851075 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:05.854575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d67a6-9485-4e8f-a429-0749274a19cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:05.856419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d67a6-9485-4e8f-a429-0749274a19cb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:05.857075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d67a6-9485-4e8f-a429-0749274a19cb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:05.857563 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d67a6-9485-4e8f-a429-0749274a19cb None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:05.858429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d67a6-9485-4e8f-a429-0749274a19cb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:05.859018 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d67a6-9485-4e8f-a429-0749274a19cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:05.861673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d67a6-9485-4e8f-a429-0749274a19cb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:05.868740 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 357/718] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 24 15:14:05 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:05.982523 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a6f36-9ca8-4630-b466-eb046230c8d6 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:05.983479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a6f36-9ca8-4630-b466-eb046230c8d6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:05.984217 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26a6f36-9ca8-4630-b466-eb046230c8d6 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:05.990892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 358/719] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:05 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:05.997007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e037b-f06e-4309-b2fb-30dfc6150264 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:05.997745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e037b-f06e-4309-b2fb-30dfc6150264 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:05.998301 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896e037b-f06e-4309-b2fb-30dfc6150264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:06.045866 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df76ff3-6af0-4239-84ca-852c68f9e039 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:06.047354 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-896e037b-f06e-4309-b2fb-30dfc6150264 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:06.048806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-896e037b-f06e-4309-b2fb-30dfc6150264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:06.118022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb6251-d26d-40cf-aca5-c9e9ad6ce3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:06.119103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb6251-d26d-40cf-aca5-c9e9ad6ce3e1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:06.119578 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb6251-d26d-40cf-aca5-c9e9ad6ce3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:06.120046 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb6251-d26d-40cf-aca5-c9e9ad6ce3e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:06.133597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb6251-d26d-40cf-aca5-c9e9ad6ce3e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:06.136574 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdb6251-d26d-40cf-aca5-c9e9ad6ce3e1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:06.152196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 359/720] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 15:14:06.160676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ccdb6251-d26d-40cf-aca5-c9e9ad6ce3e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:06.160676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:06.160676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:06.160676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:06.160676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:06.160676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:06.160676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:06.160676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:06.160676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:06.160676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:06.160676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:06.160676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:06.160676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:06.160676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:06.160676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:06.162146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:06.163092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:06.163092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:06.163092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:06.163092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:06.163092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:06.163092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:06.163092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:06.163092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:06.164015 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 362/721] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:06.169279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e037b-f06e-4309-b2fb-30dfc6150264 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:06.236234 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb6251-d26d-40cf-aca5-c9e9ad6ce3e1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:06.325320 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896e037b-f06e-4309-b2fb-30dfc6150264 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:06.346736 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:06.350600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:06.397285 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d05d84-3fae-4cc2-9dfe-6907b7040487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:06.397879 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d05d84-3fae-4cc2-9dfe-6907b7040487 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:06.398154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d05d84-3fae-4cc2-9dfe-6907b7040487 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:06.398466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d05d84-3fae-4cc2-9dfe-6907b7040487 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:06.399161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d05d84-3fae-4cc2-9dfe-6907b7040487 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:06.399462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d05d84-3fae-4cc2-9dfe-6907b7040487 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:06.401142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d05d84-3fae-4cc2-9dfe-6907b7040487 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:06.421741 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdb6251-d26d-40cf-aca5-c9e9ad6ce3e1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:06.425530 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 360/722] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:06.435303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d05d84-3fae-4cc2-9dfe-6907b7040487 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:06.493700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28de56-0ae4-448e-a199-547989d681c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:06.494511 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28de56-0ae4-448e-a199-547989d681c1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:06.494790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28de56-0ae4-448e-a199-547989d681c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:06.495064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f28de56-0ae4-448e-a199-547989d681c1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:06.507327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f28de56-0ae4-448e-a199-547989d681c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:06.512726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-7f28de56-0ae4-448e-a199-547989d681c1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:14:06.512726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:06.512726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:06.512726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:06.512726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:06.512726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:06.512726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:06.512726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:06.512726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:06.512726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:06.512726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:06.512726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:06.512726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:06.512726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:06.512726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:06.512726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:06.513637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:06.513637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:06.513637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:06.513637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:06.513637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:06.513637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:06.513637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:06.513637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:06.513637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:06.513637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:06.513637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:06.513637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:06.513637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:06.513637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:06.513637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:06.513637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:06.514904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:06.514904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:06.514904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:06.514904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:06.514904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:06.514904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:06.514904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:06.514904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:06.514904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:06.514904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:06.514904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:14:06.514904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:06.514904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 363/723] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:14:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:06.515588 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28de56-0ae4-448e-a199-547989d681c1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:06.579031 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d05d84-3fae-4cc2-9dfe-6907b7040487 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:06.619484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0b580-e8ad-4f01-b286-aaffc1ba2744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:06.621184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0b580-e8ad-4f01-b286-aaffc1ba2744 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:06.621882 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0b580-e8ad-4f01-b286-aaffc1ba2744 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:06.622529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca0b580-e8ad-4f01-b286-aaffc1ba2744 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:06.623468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0b580-e8ad-4f01-b286-aaffc1ba2744 None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=8a5fd31a2db944f596912e4f0aa26835) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:06.624059 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0b580-e8ad-4f01-b286-aaffc1ba2744 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:06.625780 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca0b580-e8ad-4f01-b286-aaffc1ba2744 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:06.638513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f28de56-0ae4-448e-a199-547989d681c1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:06.638946 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 361/724] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:14:06 2022] GET /identity/v3/role_assignments?user.id=8a5fd31a2db944f596912e4f0aa26835&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:06.640227 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0b580-e8ad-4f01-b286-aaffc1ba2744 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:06.668341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:06.689183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:06.694992 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5549e2-5d80-4a1f-86e2-2f462c971fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:06.695648 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5549e2-5d80-4a1f-86e2-2f462c971fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:06.695929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5549e2-5d80-4a1f-86e2-2f462c971fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:06.696196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5549e2-5d80-4a1f-86e2-2f462c971fa3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:06.719888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5549e2-5d80-4a1f-86e2-2f462c971fa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:06.724861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-8b5549e2-5d80-4a1f-86e2-2f462c971fa3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:06.724861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:06.724861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:06.724861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:06.724861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:06.724861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:06.724861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:06.724861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:06.724861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:06.724861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:06.724861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:06.724861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:06.724861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:06.724861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:06.724861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:06.724861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:06.727038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:06.727038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:06.727038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:06.727038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:06.727038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:06.727038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:06.727038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:06.727038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:06.727038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:06.727038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:06.727038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:06.727038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:06.727038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:06.727038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:06.727038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:06.727038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:06.729129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:06.729129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:06.729129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:06.729129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:06.729129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:06.729129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:06.729129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:06.729129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:06.729129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:06.729129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:06.729129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:06.729129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:06.734739 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 364/725] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:14:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:06.748535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5549e2-5d80-4a1f-86e2-2f462c971fa3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:06.789535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca0b580-e8ad-4f01-b286-aaffc1ba2744 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:06.846530 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b27c2-f68b-4f6c-b30e-627b768b01c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:06.849007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b27c2-f68b-4f6c-b30e-627b768b01c3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:06.849293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b27c2-f68b-4f6c-b30e-627b768b01c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:06.849567 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b27c2-f68b-4f6c-b30e-627b768b01c3 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:06.851109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b27c2-f68b-4f6c-b30e-627b768b01c3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:06.851436 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b27c2-f68b-4f6c-b30e-627b768b01c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:06.853198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b27c2-f68b-4f6c-b30e-627b768b01c3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:06.880529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 362/726] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:14:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:06.886931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b27c2-f68b-4f6c-b30e-627b768b01c3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:07.009145 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5549e2-5d80-4a1f-86e2-2f462c971fa3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:07.078777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb464dd-4cdd-475e-b76c-5b32a7ab2462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:07.079829 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb464dd-4cdd-475e-b76c-5b32a7ab2462 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:07.080286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb464dd-4cdd-475e-b76c-5b32a7ab2462 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:07.080710 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb464dd-4cdd-475e-b76c-5b32a7ab2462 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:07.081805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb464dd-4cdd-475e-b76c-5b32a7ab2462 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:07.082347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb464dd-4cdd-475e-b76c-5b32a7ab2462 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:07.085826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb464dd-4cdd-475e-b76c-5b32a7ab2462 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:07.100917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 365/727] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:14:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:07.110063 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb464dd-4cdd-475e-b76c-5b32a7ab2462 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:07.146670 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:07.178996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b27c2-f68b-4f6c-b30e-627b768b01c3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:07.230732 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:07.283765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d246a57-032b-48e2-bc2e-21947b6955d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:07.284325 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d246a57-032b-48e2-bc2e-21947b6955d7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:07.284602 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d246a57-032b-48e2-bc2e-21947b6955d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:07.284867 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d246a57-032b-48e2-bc2e-21947b6955d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:07.310555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d246a57-032b-48e2-bc2e-21947b6955d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:07.312014 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d246a57-032b-48e2-bc2e-21947b6955d7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:07.332174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0d246a57-032b-48e2-bc2e-21947b6955d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:07.332174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:07.332174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:07.332174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:07.332174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:07.332174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:07.332174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:07.332174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:07.332174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:07.332174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:07.332174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:07.332174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:07.332174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:07.332174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:07.332174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:07.332174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:07.334544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:07.336329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:07.336329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:07.336329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:07.339724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 363/728] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 448 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:07.344103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb464dd-4cdd-475e-b76c-5b32a7ab2462 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:07.346149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d246a57-032b-48e2-bc2e-21947b6955d7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:07.401491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a54659-a9d6-4634-9ac2-3c48bc652eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:07.402209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a54659-a9d6-4634-9ac2-3c48bc652eec None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:07.402508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a54659-a9d6-4634-9ac2-3c48bc652eec None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:07.402909 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a54659-a9d6-4634-9ac2-3c48bc652eec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:07.428704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a54659-a9d6-4634-9ac2-3c48bc652eec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:07.430163 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a54659-a9d6-4634-9ac2-3c48bc652eec None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:07.459552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-41a54659-a9d6-4634-9ac2-3c48bc652eec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:07.459552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:07.459552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:07.459552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:07.459552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:07.459552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:07.459552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:07.459552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:07.459552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:07.459552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:07.459552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:07.459552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:07.459552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:07.459552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:07.459552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:07.459552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:07.461469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:07.463380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:07.463380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:07.463380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:07.466391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 366/729] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 357 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:07.475088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a54659-a9d6-4634-9ac2-3c48bc652eec None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:07.554360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d246a57-032b-48e2-bc2e-21947b6955d7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:07.614572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e81814-4479-4024-b772-a043d750224a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:07.615540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e81814-4479-4024-b772-a043d750224a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:07.615983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e81814-4479-4024-b772-a043d750224a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:07.616884 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e81814-4479-4024-b772-a043d750224a None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:07.618343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e81814-4479-4024-b772-a043d750224a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:07.618878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e81814-4479-4024-b772-a043d750224a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:07.621413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e81814-4479-4024-b772-a043d750224a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:07.645060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 364/730] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:07.651793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e81814-4479-4024-b772-a043d750224a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:07.690596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:07.742856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a54659-a9d6-4634-9ac2-3c48bc652eec None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:07.757382 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:07.814317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e335a8-127f-4337-ab4c-7fc8185f8127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:07.814859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e335a8-127f-4337-ab4c-7fc8185f8127 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:07.815142 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e335a8-127f-4337-ab4c-7fc8185f8127 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:07.815415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e335a8-127f-4337-ab4c-7fc8185f8127 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:07.816070 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e335a8-127f-4337-ab4c-7fc8185f8127 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:07.821670 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e335a8-127f-4337-ab4c-7fc8185f8127 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:07.823183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e335a8-127f-4337-ab4c-7fc8185f8127 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:07.845512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 367/731] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:07.852909 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e335a8-127f-4337-ab4c-7fc8185f8127 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:07.878253 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e81814-4479-4024-b772-a043d750224a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:07.955979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a069a-3cf3-46bb-adf9-413ad1b8ff67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:07.957009 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a069a-3cf3-46bb-adf9-413ad1b8ff67 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:07.958553 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a069a-3cf3-46bb-adf9-413ad1b8ff67 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:07.959317 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776a069a-3cf3-46bb-adf9-413ad1b8ff67 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:07.968661 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a069a-3cf3-46bb-adf9-413ad1b8ff67 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:07.970697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776a069a-3cf3-46bb-adf9-413ad1b8ff67 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:07.996406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-776a069a-3cf3-46bb-adf9-413ad1b8ff67 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:14:07.996406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:07.996406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:07.996406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:07.996406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:07.996406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:07.996406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:07.996406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:07.996406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:07.996406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:07.996406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:07.996406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:07.996406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:07.996406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:07.996406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:07.996406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:07.997413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:07.998379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:07.998379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:07.998379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:07.998379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:07.998379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:07.998379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:07.998379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:07.998379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:07.998379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:14:07.998379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:08.001576 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 365/732] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 15:14:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:08.007788 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a069a-3cf3-46bb-adf9-413ad1b8ff67 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:08.037630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e335a8-127f-4337-ab4c-7fc8185f8127 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:08.077078 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f8be38-dd6b-4556-a15f-58186ccd617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:08.078162 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8be38-dd6b-4556-a15f-58186ccd617e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:08.078719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8be38-dd6b-4556-a15f-58186ccd617e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:08.079264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f8be38-dd6b-4556-a15f-58186ccd617e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:08.088270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8be38-dd6b-4556-a15f-58186ccd617e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:08.090012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f8be38-dd6b-4556-a15f-58186ccd617e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:08.100704 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:08.104773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-29f8be38-dd6b-4556-a15f-58186ccd617e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:14:08.104773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:08.104773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:08.104773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:08.104773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:08.104773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:08.104773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:08.104773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:08.104773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:08.104773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:08.104773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:08.104773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:08.104773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:08.104773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:08.104773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:08.104773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:08.105843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:08.106876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:08.106876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:08.106876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:08.106876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:08.106876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:08.106876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:08.106876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:08.106876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:08.106876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:14:08.106876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:08.110089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 368/733] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:14:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:08.111631 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc3cb9-9473-4f92-9871-1cdd3c17fbb1 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:08.111999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc3cb9-9473-4f92-9871-1cdd3c17fbb1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:08.112370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cc3cb9-9473-4f92-9871-1cdd3c17fbb1 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:08.114263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 369/734] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:08.118780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc3cb9-9473-4f92-9871-1cdd3c17fbb1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:08.146207 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:08.164677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776a069a-3cf3-46bb-adf9-413ad1b8ff67 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:08.243831 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cc3cb9-9473-4f92-9871-1cdd3c17fbb1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:08.244857 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ef6f27-62d7-4160-95a5-fac4619a6b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:08.247112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef6f27-62d7-4160-95a5-fac4619a6b5d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:08.247616 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef6f27-62d7-4160-95a5-fac4619a6b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:08.248117 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ef6f27-62d7-4160-95a5-fac4619a6b5d None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:08.249059 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef6f27-62d7-4160-95a5-fac4619a6b5d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:08.249576 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef6f27-62d7-4160-95a5-fac4619a6b5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:08.251789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ef6f27-62d7-4160-95a5-fac4619a6b5d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:08.280433 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 366/735] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 24 15:14:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:08.282560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7550133-00ac-4fcd-92b8-5a2934de1302 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:08.283154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7550133-00ac-4fcd-92b8-5a2934de1302 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:08.283747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7550133-00ac-4fcd-92b8-5a2934de1302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:08.284088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae9887e-7231-4260-9156-b1b5b1736ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:08.284977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9887e-7231-4260-9156-b1b5b1736ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:08.285258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9887e-7231-4260-9156-b1b5b1736ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:08.285529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae9887e-7231-4260-9156-b1b5b1736ee2 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:08.286336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae9887e-7231-4260-9156-b1b5b1736ee2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:08.286641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae9887e-7231-4260-9156-b1b5b1736ee2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:08.288170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae9887e-7231-4260-9156-b1b5b1736ee2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:08.308489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 370/736] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 24 15:14:08 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:08.309492 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e7550133-00ac-4fcd-92b8-5a2934de1302 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:08.309609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae9887e-7231-4260-9156-b1b5b1736ee2 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:08.311198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e7550133-00ac-4fcd-92b8-5a2934de1302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:08.399483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:08.415279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 367/737] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:14:08.415279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7550133-00ac-4fcd-92b8-5a2934de1302 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:08.437360 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae9887e-7231-4260-9156-b1b5b1736ee2 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:08.527899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:08.559338 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9976b24-9dad-4b6e-a69b-3b787185fc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:08.560556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9976b24-9dad-4b6e-a69b-3b787185fc12 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:08.561120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9976b24-9dad-4b6e-a69b-3b787185fc12 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:08.561591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9976b24-9dad-4b6e-a69b-3b787185fc12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:08.588159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7550133-00ac-4fcd-92b8-5a2934de1302 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:08.627321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82954555-624f-4bb2-b2bf-a48dda3c5bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:08.628800 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82954555-624f-4bb2-b2bf-a48dda3c5bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:08.629280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82954555-624f-4bb2-b2bf-a48dda3c5bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:08.630298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82954555-624f-4bb2-b2bf-a48dda3c5bc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:08.705994 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9976b24-9dad-4b6e-a69b-3b787185fc12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:08.708607 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9976b24-9dad-4b6e-a69b-3b787185fc12 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:08.718176 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82954555-624f-4bb2-b2bf-a48dda3c5bc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:08.720047 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82954555-624f-4bb2-b2bf-a48dda3c5bc2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:08.781988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:08.803226 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:08.827299 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 371/738] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:14:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:14:08.828963 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b158df1-5692-4828-9f54-d712645a0e17 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:08.829497 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b158df1-5692-4828-9f54-d712645a0e17 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:08.830020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b158df1-5692-4828-9f54-d712645a0e17 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:08.831914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 372/739] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:08.832855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b158df1-5692-4828-9f54-d712645a0e17 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:08.850459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 368/740] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:14:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 15:14:08.851727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d538a22-ba7f-43d9-8608-435f73efe856 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:08.852013 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d538a22-ba7f-43d9-8608-435f73efe856 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:08.852388 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d538a22-ba7f-43d9-8608-435f73efe856 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:08.858289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 369/741] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:08.858289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2576d1f-c63b-4b8c-8b1c-36abd9b84caf None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:08.858289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2576d1f-c63b-4b8c-8b1c-36abd9b84caf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:08.858289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2576d1f-c63b-4b8c-8b1c-36abd9b84caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:08.875767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d2576d1f-c63b-4b8c-8b1c-36abd9b84caf None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:08.876972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d2576d1f-c63b-4b8c-8b1c-36abd9b84caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:08.947296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 370/742] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:14:08.948272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2576d1f-c63b-4b8c-8b1c-36abd9b84caf None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:08.965350 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b158df1-5692-4828-9f54-d712645a0e17 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:09.014729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a75bd1b-70e9-4ae1-b1ff-ab9167bbbfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:09.015351 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75bd1b-70e9-4ae1-b1ff-ab9167bbbfeb None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:09.015623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75bd1b-70e9-4ae1-b1ff-ab9167bbbfeb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:09.015887 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a75bd1b-70e9-4ae1-b1ff-ab9167bbbfeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:09.029801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a75bd1b-70e9-4ae1-b1ff-ab9167bbbfeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:09.031315 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a75bd1b-70e9-4ae1-b1ff-ab9167bbbfeb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:09.034116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:09.047526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-5a75bd1b-70e9-4ae1-b1ff-ab9167bbbfeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:09.047526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:09.047526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:09.047526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:09.047526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:09.047526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:09.047526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:09.047526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:09.047526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:09.047526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:09.047526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:09.047526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:09.047526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:09.047526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:09.047526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:09.047526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:09.048976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:09.049930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:09.049930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:09.049930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:09.049930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 373/743] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:09.049930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e6536-8130-4a22-bd81-8ffcfbb52f6c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:09.050212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e6536-8130-4a22-bd81-8ffcfbb52f6c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:09.050447 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e6536-8130-4a22-bd81-8ffcfbb52f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:09.062982 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2576d1f-c63b-4b8c-8b1c-36abd9b84caf None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:09.069022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-951e6536-8130-4a22-bd81-8ffcfbb52f6c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:09.070248 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-951e6536-8130-4a22-bd81-8ffcfbb52f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:09.097085 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:09.102869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836993b-5b60-40bd-813b-b2aad11f4bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:09.103503 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836993b-5b60-40bd-813b-b2aad11f4bab None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:09.103784 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836993b-5b60-40bd-813b-b2aad11f4bab None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:09.104054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836993b-5b60-40bd-813b-b2aad11f4bab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:09.114773 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836993b-5b60-40bd-813b-b2aad11f4bab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:09.116747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5836993b-5b60-40bd-813b-b2aad11f4bab None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:09.133851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5836993b-5b60-40bd-813b-b2aad11f4bab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:09.133851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:09.133851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:09.133851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:09.133851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:09.133851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:09.133851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:09.133851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:09.133851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:09.133851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:09.133851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:09.133851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:09.133851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:09.133851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:09.133851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:09.133851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:09.134841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:09.135719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:09.135719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:09.135719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:09.139581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 371/744] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:09.140399 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836993b-5b60-40bd-813b-b2aad11f4bab None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:09.150279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 374/745] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 15:14:09.150279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e6536-8130-4a22-bd81-8ffcfbb52f6c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:09.262747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836993b-5b60-40bd-813b-b2aad11f4bab None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:09.271332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e6536-8130-4a22-bd81-8ffcfbb52f6c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:09.295706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:09.304795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d490e74-2352-4f8b-9a11-9669492299ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:09.305412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d490e74-2352-4f8b-9a11-9669492299ba None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:09.305691 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d490e74-2352-4f8b-9a11-9669492299ba None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:09.305957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d490e74-2352-4f8b-9a11-9669492299ba None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:09.306643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d490e74-2352-4f8b-9a11-9669492299ba None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:09.308031 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d490e74-2352-4f8b-9a11-9669492299ba None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:09.311867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c1026-c229-43fa-8d3a-6f76d1a3f02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:09.312475 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c1026-c229-43fa-8d3a-6f76d1a3f02c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:09.312755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c1026-c229-43fa-8d3a-6f76d1a3f02c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:09.313025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c1026-c229-43fa-8d3a-6f76d1a3f02c None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:09.313664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c1026-c229-43fa-8d3a-6f76d1a3f02c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:09.313982 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c1026-c229-43fa-8d3a-6f76d1a3f02c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:09.315719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c1026-c229-43fa-8d3a-6f76d1a3f02c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:09.326830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 372/746] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 15:14:09 2022] POST /identity/v3/services => generated 240 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:14:09.327470 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d490e74-2352-4f8b-9a11-9669492299ba None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:09.344464 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 375/747] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:09.344464 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c1026-c229-43fa-8d3a-6f76d1a3f02c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:09.388171 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:09.503200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d490e74-2352-4f8b-9a11-9669492299ba None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:09.505191 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c1026-c229-43fa-8d3a-6f76d1a3f02c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:09.557359 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8d108-7739-4048-ba3c-9fa1d5d4d90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:09.558419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8d108-7739-4048-ba3c-9fa1d5d4d90f None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:09.558901 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8d108-7739-4048-ba3c-9fa1d5d4d90f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:09.559468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8d108-7739-4048-ba3c-9fa1d5d4d90f None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:09.560575 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8d108-7739-4048-ba3c-9fa1d5d4d90f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:09.563169 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8d108-7739-4048-ba3c-9fa1d5d4d90f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:09.563356 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec465d8-2ad7-49a3-beb5-8b8bad5677cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:09.563951 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec465d8-2ad7-49a3-beb5-8b8bad5677cf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:09.564233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec465d8-2ad7-49a3-beb5-8b8bad5677cf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:09.564509 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec465d8-2ad7-49a3-beb5-8b8bad5677cf None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:09.565163 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec465d8-2ad7-49a3-beb5-8b8bad5677cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:09.565468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec465d8-2ad7-49a3-beb5-8b8bad5677cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:09.566881 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec465d8-2ad7-49a3-beb5-8b8bad5677cf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:09.587480 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 373/748] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:09.588674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec465d8-2ad7-49a3-beb5-8b8bad5677cf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:09.596542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 376/749] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 15:14:09 2022] POST /identity/v3/services => generated 251 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:14:09.596542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8d108-7739-4048-ba3c-9fa1d5d4d90f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:09.673436 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:09.733358 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:09.774992 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec465d8-2ad7-49a3-beb5-8b8bad5677cf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:09.779081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8d108-7739-4048-ba3c-9fa1d5d4d90f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:09.844986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ce043-18da-46b8-bad8-e1aa2492ad57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:09.845587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ce043-18da-46b8-bad8-e1aa2492ad57 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:09.845861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ce043-18da-46b8-bad8-e1aa2492ad57 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:09.846326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ce043-18da-46b8-bad8-e1aa2492ad57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:09.857539 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9feb4-d26c-4d7a-8130-78e32defed6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:09.858619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9feb4-d26c-4d7a-8130-78e32defed6d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:09.859002 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9feb4-d26c-4d7a-8130-78e32defed6d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:09.859874 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee9feb4-d26c-4d7a-8130-78e32defed6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:09.861186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ce043-18da-46b8-bad8-e1aa2492ad57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:09.869961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-c54ce043-18da-46b8-bad8-e1aa2492ad57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:09.869961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:09.869961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:09.869961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:09.869961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:09.869961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:09.869961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:09.869961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:09.869961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:09.869961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:09.869961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:09.869961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:09.869961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:09.869961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:09.869961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:09.869961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:09.872211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:09.872211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:09.872211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:09.872211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:09.872211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:09.872211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:09.872211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:09.872211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:09.872211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:09.872211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:09.872211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:09.872211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:09.872211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:09.872211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:09.872211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:09.872211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:09.878096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:09.878096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:09.878096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:09.878096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:09.878096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:09.878096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:09.878096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:09.878096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:09.878096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:09.878096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:09.878096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:09.878096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:09.880293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9feb4-d26c-4d7a-8130-78e32defed6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:09.880293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee9feb4-d26c-4d7a-8130-78e32defed6d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:09.880565 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 377/750] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:14:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:09.880565 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ce043-18da-46b8-bad8-e1aa2492ad57 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:09.893056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-aee9feb4-d26c-4d7a-8130-78e32defed6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:09.893056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:09.893056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:09.893056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:09.893056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:09.893056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:09.893056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:09.893056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:09.893056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:09.893056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:09.893056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:09.893056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:09.893056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:09.893056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:09.893056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:09.895859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:09.898551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:09.898551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:09.898551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:09.898551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:09.898551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:09.898551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:09.898551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:09.898551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:09.898551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 374/751] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:09.898551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9feb4-d26c-4d7a-8130-78e32defed6d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:10.024727 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ce043-18da-46b8-bad8-e1aa2492ad57 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:10.065083 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee9feb4-d26c-4d7a-8130-78e32defed6d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:10.075160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191ec21-7355-4bb1-96c7-2d4351aaf68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:10.075979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191ec21-7355-4bb1-96c7-2d4351aaf68d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:10.076342 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191ec21-7355-4bb1-96c7-2d4351aaf68d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:10.076879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191ec21-7355-4bb1-96c7-2d4351aaf68d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:10.077354 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:10.089492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191ec21-7355-4bb1-96c7-2d4351aaf68d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:10.091910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191ec21-7355-4bb1-96c7-2d4351aaf68d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:10.094073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:10.108834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-e191ec21-7355-4bb1-96c7-2d4351aaf68d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:10.108834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:10.108834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:10.108834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:10.108834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:10.108834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:10.108834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:10.108834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:10.108834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:10.108834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:10.108834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:10.108834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:10.108834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:10.108834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:10.108834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:10.109750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:10.110851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:10.110851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:10.110851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:10.110851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:10.110851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:10.110851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:10.110851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:10.110851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:10.110851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 378/752] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:10.111336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191ec21-7355-4bb1-96c7-2d4351aaf68d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:10.121783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01637f47-d26d-4cac-a09d-adaa458af4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:10.123334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01637f47-d26d-4cac-a09d-adaa458af4d4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:10.124201 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01637f47-d26d-4cac-a09d-adaa458af4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:10.125033 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01637f47-d26d-4cac-a09d-adaa458af4d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:10.126679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01637f47-d26d-4cac-a09d-adaa458af4d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:10.127506 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01637f47-d26d-4cac-a09d-adaa458af4d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:10.130682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01637f47-d26d-4cac-a09d-adaa458af4d4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:10.147055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 375/753] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:14:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:10.148922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01637f47-d26d-4cac-a09d-adaa458af4d4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:10.254629 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191ec21-7355-4bb1-96c7-2d4351aaf68d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:10.270569 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01637f47-d26d-4cac-a09d-adaa458af4d4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:10.308086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71dfd19-eab3-4301-9012-4929f5b82005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:10.308762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71dfd19-eab3-4301-9012-4929f5b82005 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:10.309042 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71dfd19-eab3-4301-9012-4929f5b82005 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:10.309301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71dfd19-eab3-4301-9012-4929f5b82005 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:10.310006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71dfd19-eab3-4301-9012-4929f5b82005 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:10.310340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71dfd19-eab3-4301-9012-4929f5b82005 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:10.311916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71dfd19-eab3-4301-9012-4929f5b82005 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:10.314475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206a8d6-491d-4069-89b3-885a5fc789f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:10.315337 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206a8d6-491d-4069-89b3-885a5fc789f6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:10.315891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206a8d6-491d-4069-89b3-885a5fc789f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:10.316545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206a8d6-491d-4069-89b3-885a5fc789f6 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:10.317808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206a8d6-491d-4069-89b3-885a5fc789f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:10.318595 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206a8d6-491d-4069-89b3-885a5fc789f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:10.321163 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1206a8d6-491d-4069-89b3-885a5fc789f6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:10.331582 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 379/754] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:10.332623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71dfd19-eab3-4301-9012-4929f5b82005 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:10.336836 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 376/755] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:10.337962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206a8d6-491d-4069-89b3-885a5fc789f6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:10.377245 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:10.427945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:10.456746 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206a8d6-491d-4069-89b3-885a5fc789f6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:10.496343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442081fc-567c-436a-8974-9dcf1a0e55ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:10.497378 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442081fc-567c-436a-8974-9dcf1a0e55ae None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:10.497867 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442081fc-567c-436a-8974-9dcf1a0e55ae None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:10.498380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442081fc-567c-436a-8974-9dcf1a0e55ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:10.499318 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442081fc-567c-436a-8974-9dcf1a0e55ae None admin] RBAC: Adding query filter params (role.id=59a2b50693a646fc8332b835233486ce, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=ad682eade4104a799b21f47aa7cdd4f5) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:10.499850 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442081fc-567c-436a-8974-9dcf1a0e55ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:10.501502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442081fc-567c-436a-8974-9dcf1a0e55ae None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:10.508129 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 377/756] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:14:10 2022] GET /identity/v3/role_assignments?user.id=ad682eade4104a799b21f47aa7cdd4f5&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=59a2b50693a646fc8332b835233486ce => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:10.509276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442081fc-567c-436a-8974-9dcf1a0e55ae None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:10.521651 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71dfd19-eab3-4301-9012-4929f5b82005 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:10.619421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf5204-e520-41e9-b4d6-571ff5621b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:10.620415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf5204-e520-41e9-b4d6-571ff5621b1b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:10.620689 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf5204-e520-41e9-b4d6-571ff5621b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:10.620956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf5204-e520-41e9-b4d6-571ff5621b1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:10.637369 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf5204-e520-41e9-b4d6-571ff5621b1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:10.639494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf5204-e520-41e9-b4d6-571ff5621b1b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:10.668931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-daaf5204-e520-41e9-b4d6-571ff5621b1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:10.668931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:10.668931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:10.668931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:10.668931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:10.668931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:10.668931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:10.668931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:10.668931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:10.668931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:10.668931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:10.668931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:10.668931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:10.668931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:10.668931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:10.668931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:10.670895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:10.672700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:10.672700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:10.672700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:10.673738 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 380/757] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:10.679591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf5204-e520-41e9-b4d6-571ff5621b1b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:10.808537 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:10.825052 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442081fc-567c-436a-8974-9dcf1a0e55ae None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:10.916832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773549c-0acf-4dc9-907e-396923918203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:10.919293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773549c-0acf-4dc9-907e-396923918203 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:10.920183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773549c-0acf-4dc9-907e-396923918203 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:10.920949 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773549c-0acf-4dc9-907e-396923918203 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:10.922619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773549c-0acf-4dc9-907e-396923918203 None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=0679072b37db41e880a95b738bea47ea) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:10.923504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773549c-0acf-4dc9-907e-396923918203 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:10.927514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773549c-0acf-4dc9-907e-396923918203 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:10.939000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 378/758] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:14:10 2022] GET /identity/v3/role_assignments?user.id=0679072b37db41e880a95b738bea47ea&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 259 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:10.979565 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf5204-e520-41e9-b4d6-571ff5621b1b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:11.003273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:11.040841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37dab4-4175-4d92-9f53-762486e9a712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:11.041473 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37dab4-4175-4d92-9f53-762486e9a712 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:11.041752 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37dab4-4175-4d92-9f53-762486e9a712 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:11.042024 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd37dab4-4175-4d92-9f53-762486e9a712 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:11.042714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37dab4-4175-4d92-9f53-762486e9a712 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:11.043024 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37dab4-4175-4d92-9f53-762486e9a712 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:11.044722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd37dab4-4175-4d92-9f53-762486e9a712 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:11.069578 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 381/759] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:11.078079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773549c-0acf-4dc9-907e-396923918203 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:11.249486 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773549c-0acf-4dc9-907e-396923918203 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:11.299324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974559f-386b-4dda-ab7e-2a6db599b6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:11.300029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974559f-386b-4dda-ab7e-2a6db599b6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:11.300348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974559f-386b-4dda-ab7e-2a6db599b6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:11.300740 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974559f-386b-4dda-ab7e-2a6db599b6d0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:11.315655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974559f-386b-4dda-ab7e-2a6db599b6d0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:11.318036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974559f-386b-4dda-ab7e-2a6db599b6d0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:11.335343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2974559f-386b-4dda-ab7e-2a6db599b6d0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:14:11.335343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:11.335343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:11.335343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:11.335343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:11.335343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:11.335343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:11.335343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:11.335343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:11.335343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:11.335343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:11.335343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:11.335343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:11.335343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:11.335343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:11.335343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:11.336332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:11.337251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:11.337251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:11.337251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:11.337251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:11.337251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:11.337251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:11.337251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:11.337251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:11.337251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:14:11.337251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:11.337251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 379/760] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 15:14:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:11.342453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37dab4-4175-4d92-9f53-762486e9a712 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:11.492823 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd37dab4-4175-4d92-9f53-762486e9a712 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:11.544312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca18a8-65e6-4f15-ba33-83bb6c3beef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:11.544913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca18a8-65e6-4f15-ba33-83bb6c3beef0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:11.545188 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca18a8-65e6-4f15-ba33-83bb6c3beef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:11.545457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca18a8-65e6-4f15-ba33-83bb6c3beef0 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:11.546190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca18a8-65e6-4f15-ba33-83bb6c3beef0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:11.546514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca18a8-65e6-4f15-ba33-83bb6c3beef0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:11.547955 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daca18a8-65e6-4f15-ba33-83bb6c3beef0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:11.576647 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 382/761] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 15:14:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:11.582137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974559f-386b-4dda-ab7e-2a6db599b6d0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:11.615487 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:11.738055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974559f-386b-4dda-ab7e-2a6db599b6d0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:11.793004 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e33433-e5ef-497b-b624-0023f20828af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:11.793741 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e33433-e5ef-497b-b624-0023f20828af None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:11.794184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e33433-e5ef-497b-b624-0023f20828af None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:11.794603 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e33433-e5ef-497b-b624-0023f20828af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:11.810987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e33433-e5ef-497b-b624-0023f20828af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:11.813765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e33433-e5ef-497b-b624-0023f20828af None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:11.836591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-69e33433-e5ef-497b-b624-0023f20828af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:11.836591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:11.836591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:11.836591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:11.836591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:11.836591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:11.836591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:11.836591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:11.836591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:11.836591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:11.836591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:11.836591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:11.836591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:11.836591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:11.836591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:11.836591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:11.838055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:11.839114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:11.839114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:11.839114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:11.839114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 380/762] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:11.841284 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc70bd-cd3e-43cb-9708-2c3fa325a904 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:11.841627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc70bd-cd3e-43cb-9708-2c3fa325a904 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:11.841905 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc70bd-cd3e-43cb-9708-2c3fa325a904 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:11.843264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 383/763] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:11.845539 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e33433-e5ef-497b-b624-0023f20828af None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:11.857432 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a39db-ad92-49c1-9600-a108c23b9d8a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:11.857432 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a39db-ad92-49c1-9600-a108c23b9d8a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:11.857432 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a39db-ad92-49c1-9600-a108c23b9d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:11.878363 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-354a39db-ad92-49c1-9600-a108c23b9d8a None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:11.880741 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-354a39db-ad92-49c1-9600-a108c23b9d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:11.925201 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:11.987421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 384/764] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:12.035250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:12.094145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e33433-e5ef-497b-b624-0023f20828af None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:12.155851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fb322-6e9d-4509-8ff7-ed81b37fb521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:12.156734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fb322-6e9d-4509-8ff7-ed81b37fb521 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:12.157097 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fb322-6e9d-4509-8ff7-ed81b37fb521 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:12.157404 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fb322-6e9d-4509-8ff7-ed81b37fb521 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:12.158561 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fb322-6e9d-4509-8ff7-ed81b37fb521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:12.159089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fb322-6e9d-4509-8ff7-ed81b37fb521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:12.161815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990fb322-6e9d-4509-8ff7-ed81b37fb521 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:12.189143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b705c8-c72e-4e62-a346-59ab8055b2a0 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:12.189753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b705c8-c72e-4e62-a346-59ab8055b2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:12.190274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b705c8-c72e-4e62-a346-59ab8055b2a0 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:12.190402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 381/765] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:12.191810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fb322-6e9d-4509-8ff7-ed81b37fb521 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:12.193202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 385/766] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:14:12.200179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9853e4b-04ba-458f-8bf0-bb82938c3a47 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:12.200774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9853e4b-04ba-458f-8bf0-bb82938c3a47 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:12.201240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9853e4b-04ba-458f-8bf0-bb82938c3a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:12.231937 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d9853e4b-04ba-458f-8bf0-bb82938c3a47 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:12.234519 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d9853e4b-04ba-458f-8bf0-bb82938c3a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:12.370004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 386/767] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:12.391905 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9853e4b-04ba-458f-8bf0-bb82938c3a47 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:12.402926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fb322-6e9d-4509-8ff7-ed81b37fb521 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:12.460086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:12.470796 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9caba7-eada-414d-be57-ba295260b925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:12.471438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9caba7-eada-414d-be57-ba295260b925 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:12.471718 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9caba7-eada-414d-be57-ba295260b925 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:12.471985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9caba7-eada-414d-be57-ba295260b925 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:12.487981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9caba7-eada-414d-be57-ba295260b925 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:12.489465 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9caba7-eada-414d-be57-ba295260b925 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:12.516184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ce9caba7-eada-414d-be57-ba295260b925 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:12.516184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:12.516184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:12.516184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:12.516184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:12.516184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:12.516184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:12.516184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:12.516184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:12.516184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:12.516184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:12.516184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:12.516184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:12.516184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:12.516184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:12.517645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:12.519360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:12.519360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:12.519360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:12.519360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:12.519360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:12.519360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:12.519360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:12.519360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:12.520891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 382/768] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:12.526200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9caba7-eada-414d-be57-ba295260b925 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:12.588806 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:12.610630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9853e4b-04ba-458f-8bf0-bb82938c3a47 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:12.670983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cfbac-4dd6-4ccd-8ed1-f47a63e8d7e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:12.671660 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cfbac-4dd6-4ccd-8ed1-f47a63e8d7e5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:12.671945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cfbac-4dd6-4ccd-8ed1-f47a63e8d7e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:12.672226 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671cfbac-4dd6-4ccd-8ed1-f47a63e8d7e5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:12.672836 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671cfbac-4dd6-4ccd-8ed1-f47a63e8d7e5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:12.674246 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671cfbac-4dd6-4ccd-8ed1-f47a63e8d7e5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-671cfbac-4dd6-4ccd-8ed1-f47a63e8d7e5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:12.698517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:12.699954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:12.699954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:12.699954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:12.699954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:12.699954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:12.699954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:12.699954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:12.699954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:12.699954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:12.699954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:12.699954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:12.699954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:12.699954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:12.699954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:12.699954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:12.699954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:12.700787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:12.700787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:12.700787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:12.700787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:12.700787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:12.700787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:12.700787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:12.700787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:12.700787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:12.700787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:12.700787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:12.700787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:12.700787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:14:12.700787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:12.705749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 387/769] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:12 2022] GET /identity/v3/services/network => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:12.706157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9caba7-eada-414d-be57-ba295260b925 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:12.713433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cfbac-4dd6-4ccd-8ed1-f47a63e8d7e5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:12.762249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a25c6-4ed7-4a48-8607-fff39fb7ae93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:12.762867 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a25c6-4ed7-4a48-8607-fff39fb7ae93 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:12.763149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a25c6-4ed7-4a48-8607-fff39fb7ae93 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:12.763420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82a25c6-4ed7-4a48-8607-fff39fb7ae93 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:12.764277 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a25c6-4ed7-4a48-8607-fff39fb7ae93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:12.765348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a25c6-4ed7-4a48-8607-fff39fb7ae93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:12.766891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82a25c6-4ed7-4a48-8607-fff39fb7ae93 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:12.781688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 383/770] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:12.782770 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a25c6-4ed7-4a48-8607-fff39fb7ae93 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:12.854368 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671cfbac-4dd6-4ccd-8ed1-f47a63e8d7e5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:12.862180 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:12.914623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f6490-5d9d-4b37-9cc8-c18ee4913e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:12.915261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f6490-5d9d-4b37-9cc8-c18ee4913e5c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:12.915537 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f6490-5d9d-4b37-9cc8-c18ee4913e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:12.915804 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345f6490-5d9d-4b37-9cc8-c18ee4913e5c None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:12.916434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f6490-5d9d-4b37-9cc8-c18ee4913e5c None admin] RBAC: Adding query filter params (name=network) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:12.916734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f6490-5d9d-4b37-9cc8-c18ee4913e5c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:12.918536 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345f6490-5d9d-4b37-9cc8-c18ee4913e5c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:12.925404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 388/771] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 15:14:12 2022] GET /identity/v3/services?name=network => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:12.926558 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f6490-5d9d-4b37-9cc8-c18ee4913e5c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:12.951688 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:12.953811 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82a25c6-4ed7-4a48-8607-fff39fb7ae93 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:13.016144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3628f-3696-40df-9ab3-50ae7f6fce27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:13.018017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3628f-3696-40df-9ab3-50ae7f6fce27 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:13.018892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3628f-3696-40df-9ab3-50ae7f6fce27 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:13.019631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a3628f-3696-40df-9ab3-50ae7f6fce27 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:13.021608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3628f-3696-40df-9ab3-50ae7f6fce27 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:13.024363 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a3628f-3696-40df-9ab3-50ae7f6fce27 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-e3a3628f-3696-40df-9ab3-50ae7f6fce27 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:13.043185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:13.045121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:13.045121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:13.045121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:13.045121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:13.045121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:13.045121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:13.045121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:13.045121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:13.045121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:13.045121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:13.045121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:13.045121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:13.045121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:13.045121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:13.045121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:13.045121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:13.046807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:13.046807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:13.046807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:13.046807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:13.046807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:13.046807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:13.046807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:13.046807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:13.046807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:13.046807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:13.046807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:13.046807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:13.046807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:14:13.046807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:13.051897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 384/772] 10.222.0.76 () {56 vars in 1129 bytes} [Wed Aug 24 15:14:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:13.053811 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3628f-3696-40df-9ab3-50ae7f6fce27 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:13.063897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345f6490-5d9d-4b37-9cc8-c18ee4913e5c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:13.118583 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70750f0e-0e8d-4f7c-8385-1052afeae679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:13.119289 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70750f0e-0e8d-4f7c-8385-1052afeae679 None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:13.119561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70750f0e-0e8d-4f7c-8385-1052afeae679 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:13.119835 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70750f0e-0e8d-4f7c-8385-1052afeae679 None admin] PATH_INFO: `/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/8a5fd31a2db944f596912e4f0aa26835/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:13.122087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70750f0e-0e8d-4f7c-8385-1052afeae679 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c010096b77f4cd3b2f32086b1ba7ea7, user_id=8a5fd31a2db944f596912e4f0aa26835, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:13.123902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70750f0e-0e8d-4f7c-8385-1052afeae679 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:13.142170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 389/773] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:14:12 2022] PUT /identity/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/8a5fd31a2db944f596912e4f0aa26835/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:14:13.143186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70750f0e-0e8d-4f7c-8385-1052afeae679 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:13.203980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a3628f-3696-40df-9ab3-50ae7f6fce27 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:13.224576 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:13.255036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492f19d-6071-4d5d-80ff-37b52af25adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:13.256021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492f19d-6071-4d5d-80ff-37b52af25adf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:13.256563 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492f19d-6071-4d5d-80ff-37b52af25adf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:13.257126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a492f19d-6071-4d5d-80ff-37b52af25adf None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:13.258013 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492f19d-6071-4d5d-80ff-37b52af25adf None admin] RBAC: Adding query filter params (type=network) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:13.258565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492f19d-6071-4d5d-80ff-37b52af25adf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:13.260169 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a492f19d-6071-4d5d-80ff-37b52af25adf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:13.268252 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 385/774] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 15:14:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:13.275156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:13.277670 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492f19d-6071-4d5d-80ff-37b52af25adf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:13.311915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70750f0e-0e8d-4f7c-8385-1052afeae679 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:13.359308 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13191e77-109b-457a-a06e-e667116bd8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:13.359930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13191e77-109b-457a-a06e-e667116bd8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:13.360205 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13191e77-109b-457a-a06e-e667116bd8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:13.360472 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13191e77-109b-457a-a06e-e667116bd8b6 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:13.361641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13191e77-109b-457a-a06e-e667116bd8b6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:13.361945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13191e77-109b-457a-a06e-e667116bd8b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:13.370250 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13191e77-109b-457a-a06e-e667116bd8b6 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:13.370250 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 390/775] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 24 15:14:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:13.376263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13191e77-109b-457a-a06e-e667116bd8b6 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:13.461178 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a492f19d-6071-4d5d-80ff-37b52af25adf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:13.500351 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32323782-19f5-470c-9cea-49ef2be0856d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:13.501154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32323782-19f5-470c-9cea-49ef2be0856d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:13.501655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32323782-19f5-470c-9cea-49ef2be0856d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:13.502025 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32323782-19f5-470c-9cea-49ef2be0856d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:13.502845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32323782-19f5-470c-9cea-49ef2be0856d None admin] RBAC: Adding query filter params (interface=public, service_id=15f63e28cad649c1a546e8257d6f031c, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:13.503155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32323782-19f5-470c-9cea-49ef2be0856d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:13.505271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32323782-19f5-470c-9cea-49ef2be0856d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:13.508639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13191e77-109b-457a-a06e-e667116bd8b6 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:13.513678 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 386/776] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 24 15:14:13 2022] GET /identity/v3/endpoints?service_id=15f63e28cad649c1a546e8257d6f031c&interface=public®ion_id=RegionOne => generated 194 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:13.515034 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e0e86-78e8-44f5-9511-95e45e1a7765 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:13.515324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e0e86-78e8-44f5-9511-95e45e1a7765 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:13.515598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e0e86-78e8-44f5-9511-95e45e1a7765 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:13.517172 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 387/777] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:13.523247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76242465-fae2-4f68-8dbf-c146958a4052 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:13.523528 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76242465-fae2-4f68-8dbf-c146958a4052 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:13.523901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76242465-fae2-4f68-8dbf-c146958a4052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:13.545347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f792d-c2ef-47fc-afd4-d8611ab83e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:13.545958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f792d-c2ef-47fc-afd4-d8611ab83e8b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:13.546242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f792d-c2ef-47fc-afd4-d8611ab83e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:13.546527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f792d-c2ef-47fc-afd4-d8611ab83e8b None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:13.547213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f792d-c2ef-47fc-afd4-d8611ab83e8b None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:13.547510 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f792d-c2ef-47fc-afd4-d8611ab83e8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:13.548600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-76242465-fae2-4f68-8dbf-c146958a4052 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:13.548927 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f792d-c2ef-47fc-afd4-d8611ab83e8b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:13.549747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-76242465-fae2-4f68-8dbf-c146958a4052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:13.556337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 391/778] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 24 15:14:13 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:13.557181 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f792d-c2ef-47fc-afd4-d8611ab83e8b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:13.560743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:13.597080 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:13.638266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 388/779] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:13.638266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76242465-fae2-4f68-8dbf-c146958a4052 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:13.689730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f792d-c2ef-47fc-afd4-d8611ab83e8b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:13.739844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62007f54-45ea-4059-bc49-3b906da8a697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:13.740413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62007f54-45ea-4059-bc49-3b906da8a697 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:13.740689 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62007f54-45ea-4059-bc49-3b906da8a697 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:13.740843 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76242465-fae2-4f68-8dbf-c146958a4052 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:13.740939 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62007f54-45ea-4059-bc49-3b906da8a697 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:13.741554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62007f54-45ea-4059-bc49-3b906da8a697 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:13.744140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62007f54-45ea-4059-bc49-3b906da8a697 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:13.750596 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 392/780] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 15:14:13 2022] GET /identity/v3/services => generated 772 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:13.752830 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd826d-61af-47b1-b596-cf45a7d31cbe None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:13.753120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd826d-61af-47b1-b596-cf45a7d31cbe None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:13.753549 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd826d-61af-47b1-b596-cf45a7d31cbe None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:13.758399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 393/781] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:13.762437 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b0a3d-0a84-41d9-88fe-50b0ecf8d0e4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:13.762745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b0a3d-0a84-41d9-88fe-50b0ecf8d0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:13.762991 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53b0a3d-0a84-41d9-88fe-50b0ecf8d0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:13.782428 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f53b0a3d-0a84-41d9-88fe-50b0ecf8d0e4 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:13.783769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f53b0a3d-0a84-41d9-88fe-50b0ecf8d0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:13.803875 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ff749-1946-4753-94f4-0225b6b1c7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:13.804524 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ff749-1946-4753-94f4-0225b6b1c7de None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:13.806667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ff749-1946-4753-94f4-0225b6b1c7de None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:13.806967 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8ff749-1946-4753-94f4-0225b6b1c7de None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:13.808843 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ff749-1946-4753-94f4-0225b6b1c7de None admin] RBAC: Adding query filter params (interface=public, service_id=7197bf1dbc9546a591598cadf53110e0, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:13.809300 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ff749-1946-4753-94f4-0225b6b1c7de None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:13.813744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8ff749-1946-4753-94f4-0225b6b1c7de None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:13.828447 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 389/782] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 24 15:14:13 2022] GET /identity/v3/endpoints?service_id=7197bf1dbc9546a591598cadf53110e0&interface=public®ion_id=RegionOne => generated 194 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:13.834433 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ff749-1946-4753-94f4-0225b6b1c7de None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:13.906699 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:13.917513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:13.918782 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 394/783] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:13.931857 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b0a3d-0a84-41d9-88fe-50b0ecf8d0e4 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:14.013755 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8ff749-1946-4753-94f4-0225b6b1c7de None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:14.051862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ac364-3b3e-47fe-a883-f0c665a92cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:14.052839 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ac364-3b3e-47fe-a883-f0c665a92cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:14.053366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ac364-3b3e-47fe-a883-f0c665a92cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:14.053842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ac364-3b3e-47fe-a883-f0c665a92cd8 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:14.054699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ac364-3b3e-47fe-a883-f0c665a92cd8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:14.056262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ac364-3b3e-47fe-a883-f0c665a92cd8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:14.062287 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 390/784] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 15:14:13 2022] GET /identity/v3/services => generated 772 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:14.065460 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53b0a3d-0a84-41d9-88fe-50b0ecf8d0e4 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:14.109363 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab0683-9375-434d-90e9-1d09cc0156de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:14.110591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab0683-9375-434d-90e9-1d09cc0156de None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:14.111090 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab0683-9375-434d-90e9-1d09cc0156de None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:14.111572 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ab0683-9375-434d-90e9-1d09cc0156de None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:14.129372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ab0683-9375-434d-90e9-1d09cc0156de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:14.138514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-23ab0683-9375-434d-90e9-1d09cc0156de None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:14:14.138514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:14.138514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:14.138514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:14.138514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:14.138514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:14.138514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:14.138514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:14.138514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:14.138514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:14.138514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:14.138514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:14.138514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:14.138514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:14.138514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:14.138514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:14.139541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:14.139541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:14.139541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:14.139541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:14.139541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:14.139541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:14.139541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:14.139541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:14.139541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:14.139541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:14.139541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:14.139541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:14.139541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:14.139541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:14.139541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:14.139541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:14.140392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:14.140392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:14.140392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:14.140392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:14.140392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:14.140392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:14.140392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:14.140392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:14.140392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:14.140392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:14.140392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:14:14.140392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:14.140392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 395/785] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:14:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:14.148474 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ac364-3b3e-47fe-a883-f0c665a92cd8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:14.286620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:14.290142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ac364-3b3e-47fe-a883-f0c665a92cd8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:14.304645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab0683-9375-434d-90e9-1d09cc0156de None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:14.334508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce094ae-18c4-4a85-b048-eb9cd9a308aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:14.335417 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce094ae-18c4-4a85-b048-eb9cd9a308aa None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:14.335697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce094ae-18c4-4a85-b048-eb9cd9a308aa None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:14.335965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce094ae-18c4-4a85-b048-eb9cd9a308aa None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:14.337106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce094ae-18c4-4a85-b048-eb9cd9a308aa None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:14.337462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce094ae-18c4-4a85-b048-eb9cd9a308aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:14.338856 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce094ae-18c4-4a85-b048-eb9cd9a308aa None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:14.356701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 391/786] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:14:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:14.359248 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce094ae-18c4-4a85-b048-eb9cd9a308aa None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:14.387784 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:14.435579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ab0683-9375-434d-90e9-1d09cc0156de None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:14.476124 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ffeb6-91cc-4f50-ae06-edc265780ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:14.476764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ffeb6-91cc-4f50-ae06-edc265780ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:14.477042 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ffeb6-91cc-4f50-ae06-edc265780ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:14.477310 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ffeb6-91cc-4f50-ae06-edc265780ef9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:14.489993 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ffeb6-91cc-4f50-ae06-edc265780ef9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:14.495128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-5f1ffeb6-91cc-4f50-ae06-edc265780ef9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:14.495128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:14.495128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:14.495128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:14.495128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:14.495128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:14.495128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:14.495128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:14.495128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:14.495128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:14.495128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:14.495128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:14.495128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:14.495128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:14.495128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:14.495128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:14.496059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:14.496059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:14.496059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:14.496059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:14.496059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:14.496059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:14.496059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:14.496059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:14.496059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:14.496059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:14.496059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:14.496059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:14.496059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:14.496059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:14.496059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:14.496059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:14.496926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:14.496926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:14.496926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:14.496926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:14.496926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:14.496926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:14.496926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:14.496926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:14.496926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:14.496926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:14.496926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:14.496926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:14.496926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 396/787] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:14:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:14.504640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ffeb6-91cc-4f50-ae06-edc265780ef9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:14.544285 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce094ae-18c4-4a85-b048-eb9cd9a308aa None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:14.615015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e0314-94bf-4f88-b025-3e58741ce833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:14.615668 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e0314-94bf-4f88-b025-3e58741ce833 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:14.615951 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e0314-94bf-4f88-b025-3e58741ce833 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:14.616227 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e0314-94bf-4f88-b025-3e58741ce833 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:14.633178 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ffeb6-91cc-4f50-ae06-edc265780ef9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:14.639912 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e0314-94bf-4f88-b025-3e58741ce833 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:14.641589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e0314-94bf-4f88-b025-3e58741ce833 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:14.653950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:14.671098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-393e0314-94bf-4f88-b025-3e58741ce833 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:14.671098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:14.671098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:14.671098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:14.671098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:14.671098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:14.671098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:14.671098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:14.671098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:14.671098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:14.671098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:14.671098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:14.671098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:14.671098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:14.671098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:14.671098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:14.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:14.673996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:14.673996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:14.673996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:14.673996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 392/788] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:14.677013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebed447-f659-4018-afc6-6ef38a88c184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:14.677013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebed447-f659-4018-afc6-6ef38a88c184 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:14.677013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebed447-f659-4018-afc6-6ef38a88c184 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:14.677013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebed447-f659-4018-afc6-6ef38a88c184 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:14.677692 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebed447-f659-4018-afc6-6ef38a88c184 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:14.677692 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebed447-f659-4018-afc6-6ef38a88c184 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:14.678439 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e0314-94bf-4f88-b025-3e58741ce833 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:14.679426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebed447-f659-4018-afc6-6ef38a88c184 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:14.690440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 397/789] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:14:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:14.696547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebed447-f659-4018-afc6-6ef38a88c184 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:14.714544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:14.862272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e0314-94bf-4f88-b025-3e58741ce833 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:14.909157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32f4a2-647f-4336-8248-04dab36d729c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:14.909157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32f4a2-647f-4336-8248-04dab36d729c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:14.909157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32f4a2-647f-4336-8248-04dab36d729c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:14.909157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32f4a2-647f-4336-8248-04dab36d729c None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:14.909595 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32f4a2-647f-4336-8248-04dab36d729c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:14.909595 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32f4a2-647f-4336-8248-04dab36d729c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:14.909595 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32f4a2-647f-4336-8248-04dab36d729c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:14.965410 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebed447-f659-4018-afc6-6ef38a88c184 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:15.055940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 393/790] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:15.059665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b11093-7b04-4b57-8198-29f45fec66ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:15.060319 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b11093-7b04-4b57-8198-29f45fec66ff None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:15.060601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b11093-7b04-4b57-8198-29f45fec66ff None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:15.060875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b11093-7b04-4b57-8198-29f45fec66ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:15.068078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32f4a2-647f-4336-8248-04dab36d729c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:15.089683 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b11093-7b04-4b57-8198-29f45fec66ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:15.091376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b11093-7b04-4b57-8198-29f45fec66ff None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:15.116395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-12b11093-7b04-4b57-8198-29f45fec66ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:15.116395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:15.116395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:15.116395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:15.116395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:15.116395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:15.116395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:15.116395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:15.116395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:15.116395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:15.116395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:15.116395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:15.116395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:15.116395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:15.116395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:15.116395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:15.117357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:15.118266 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:15.118266 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:15.118266 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:15.122504 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 398/791] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:15.124402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b11093-7b04-4b57-8198-29f45fec66ff None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:15.188988 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:15.214623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:15.239937 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32f4a2-647f-4336-8248-04dab36d729c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:15.273512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f172b0e-dec0-4f2f-b6d6-0ba535a95c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:15.274639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f172b0e-dec0-4f2f-b6d6-0ba535a95c80 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:15.274754 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f172b0e-dec0-4f2f-b6d6-0ba535a95c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:15.275030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f172b0e-dec0-4f2f-b6d6-0ba535a95c80 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:15.283558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f172b0e-dec0-4f2f-b6d6-0ba535a95c80 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:15.285028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f172b0e-dec0-4f2f-b6d6-0ba535a95c80 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:15.297700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-9f172b0e-dec0-4f2f-b6d6-0ba535a95c80 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:14:15.297700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:15.297700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:15.297700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:15.297700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:15.297700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:15.297700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:15.297700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:15.297700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:15.297700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:15.297700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:15.297700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:15.297700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:15.297700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:15.297700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:15.297700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:15.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:15.300088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:15.300088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:15.300088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:15.300088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:15.300088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:15.300088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:15.300088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:15.300088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:15.300088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:14:15.300088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:15.300088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 394/792] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 15:14:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:15.300699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011d2d4-199f-4ff1-9ad3-5629c1d6f5c0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:15.300699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011d2d4-199f-4ff1-9ad3-5629c1d6f5c0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:15.301101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2011d2d4-199f-4ff1-9ad3-5629c1d6f5c0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:15.311235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 395/793] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:15.311235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011d2d4-199f-4ff1-9ad3-5629c1d6f5c0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:15.321697 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b11093-7b04-4b57-8198-29f45fec66ff None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:15.361153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c67d6a-094e-47f6-9fc8-b85b712cccc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:15.362273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c67d6a-094e-47f6-9fc8-b85b712cccc9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:15.362768 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c67d6a-094e-47f6-9fc8-b85b712cccc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:15.363312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c67d6a-094e-47f6-9fc8-b85b712cccc9 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:15.364783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c67d6a-094e-47f6-9fc8-b85b712cccc9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:15.365384 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c67d6a-094e-47f6-9fc8-b85b712cccc9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:15.367123 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c67d6a-094e-47f6-9fc8-b85b712cccc9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:15.381447 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 399/794] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:15.383363 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9e7df-2528-409d-92a3-e045a2b4ff9d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:15.383924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9e7df-2528-409d-92a3-e045a2b4ff9d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:15.384398 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f9e7df-2528-409d-92a3-e045a2b4ff9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:15.405977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c1f9e7df-2528-409d-92a3-e045a2b4ff9d None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:15.408091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c1f9e7df-2528-409d-92a3-e045a2b4ff9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:15.421450 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2011d2d4-199f-4ff1-9ad3-5629c1d6f5c0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:15.463078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc604b-2aef-4674-a992-609f467ba6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:15.463913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc604b-2aef-4674-a992-609f467ba6ee None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:15.464026 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc604b-2aef-4674-a992-609f467ba6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:15.464244 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc604b-2aef-4674-a992-609f467ba6ee None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:15.464983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc604b-2aef-4674-a992-609f467ba6ee None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:15.465292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc604b-2aef-4674-a992-609f467ba6ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:15.466770 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc604b-2aef-4674-a992-609f467ba6ee None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:15.468725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:15.490530 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 396/795] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 24 15:14:15 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:15.491582 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc604b-2aef-4674-a992-609f467ba6ee None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:15.502389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 400/796] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:15.503929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9e7df-2528-409d-92a3-e045a2b4ff9d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:15.546071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:15.660050 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc604b-2aef-4674-a992-609f467ba6ee None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:15.714694 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f9e7df-2528-409d-92a3-e045a2b4ff9d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:15.731232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07668922-7851-47f7-9ec7-9b23d5471b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:15.731934 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07668922-7851-47f7-9ec7-9b23d5471b87 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:15.732218 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07668922-7851-47f7-9ec7-9b23d5471b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:15.732491 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07668922-7851-47f7-9ec7-9b23d5471b87 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:15.741007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07668922-7851-47f7-9ec7-9b23d5471b87 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:15.742506 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07668922-7851-47f7-9ec7-9b23d5471b87 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:15.770985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-07668922-7851-47f7-9ec7-9b23d5471b87 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:14:15.770985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:15.770985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:15.770985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:15.770985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:15.770985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:15.770985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:15.770985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:15.770985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:15.770985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:15.770985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:15.770985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:15.770985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:15.770985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:15.770985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:15.770985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:15.771979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:15.772899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:15.772899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:15.772899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:15.772899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:15.772899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:15.772899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:15.772899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:15.772899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:15.772899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:14:15.772899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:15.777591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 397/797] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:14:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:15.780075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07668922-7851-47f7-9ec7-9b23d5471b87 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:15.800796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295db607-f805-4a03-94ea-c698132a7ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:15.801493 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295db607-f805-4a03-94ea-c698132a7ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:15.801786 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295db607-f805-4a03-94ea-c698132a7ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:15.802037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295db607-f805-4a03-94ea-c698132a7ed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:15.823566 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295db607-f805-4a03-94ea-c698132a7ed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:15.825115 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295db607-f805-4a03-94ea-c698132a7ed3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:15.864913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-295db607-f805-4a03-94ea-c698132a7ed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:15.864913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:15.864913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:15.864913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:15.864913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:15.864913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:15.864913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:15.864913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:15.864913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:15.864913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:15.864913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:15.864913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:15.864913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:15.864913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:15.864913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:15.864913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:15.867133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:15.869246 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:15.869246 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:15.869246 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:15.874271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 401/798] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:15.874271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295db607-f805-4a03-94ea-c698132a7ed3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:15.887587 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:15.969254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07668922-7851-47f7-9ec7-9b23d5471b87 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:15.991341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:16.018817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5f349-a239-472f-9fcb-b93da90fdde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:16.019550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5f349-a239-472f-9fcb-b93da90fdde9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:16.020414 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5f349-a239-472f-9fcb-b93da90fdde9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:16.020751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5f349-a239-472f-9fcb-b93da90fdde9 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:16.021404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295db607-f805-4a03-94ea-c698132a7ed3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:16.021620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5f349-a239-472f-9fcb-b93da90fdde9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:16.021989 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5f349-a239-472f-9fcb-b93da90fdde9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:16.023771 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5f349-a239-472f-9fcb-b93da90fdde9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:16.049945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 398/799] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 24 15:14:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:16.051722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5f349-a239-472f-9fcb-b93da90fdde9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:16.070059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6926f6-8aa5-4ad1-a920-a4d0c6d41bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:16.070720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6926f6-8aa5-4ad1-a920-a4d0c6d41bfc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:16.070999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6926f6-8aa5-4ad1-a920-a4d0c6d41bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:16.071278 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6926f6-8aa5-4ad1-a920-a4d0c6d41bfc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:16.086006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f6926f6-8aa5-4ad1-a920-a4d0c6d41bfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:16.093858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-9f6926f6-8aa5-4ad1-a920-a4d0c6d41bfc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:16.093858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:16.093858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:16.093858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:16.093858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:16.093858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:16.093858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:16.093858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:16.093858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:16.093858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:16.093858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:16.093858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:16.093858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:16.093858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:16.093858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:16.093858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:16.095000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:16.095000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:16.095000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:16.095000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:16.095000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:16.095000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:16.095000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:16.095000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:16.095000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:16.095000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:16.095000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:16.095000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:16.095000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:16.095000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:16.095000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:16.095000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:16.095843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:16.095843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:16.095843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:16.095843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:16.095843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:16.095843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:16.095843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:16.095843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:16.095843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:16.095843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:16.095843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:16.095843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:16.095843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 402/800] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:14:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:16.096559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6926f6-8aa5-4ad1-a920-a4d0c6d41bfc None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:16.191457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5f349-a239-472f-9fcb-b93da90fdde9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:16.207321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:16.231129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6926f6-8aa5-4ad1-a920-a4d0c6d41bfc None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:16.247105 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5bc37-f3c1-4968-9997-bc577fabae03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:16.247794 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5bc37-f3c1-4968-9997-bc577fabae03 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:16.248158 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5bc37-f3c1-4968-9997-bc577fabae03 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:16.248442 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c5bc37-f3c1-4968-9997-bc577fabae03 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:16.249331 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5bc37-f3c1-4968-9997-bc577fabae03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:16.249628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5bc37-f3c1-4968-9997-bc577fabae03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:16.251266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c5bc37-f3c1-4968-9997-bc577fabae03 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:16.272882 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 399/801] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:16.272882 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5bc37-f3c1-4968-9997-bc577fabae03 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:16.276395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645fc04-ae8b-43d2-9377-8a057e61543a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:16.278109 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645fc04-ae8b-43d2-9377-8a057e61543a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:16.278511 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645fc04-ae8b-43d2-9377-8a057e61543a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:16.278894 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645fc04-ae8b-43d2-9377-8a057e61543a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:16.281392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:16.295088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645fc04-ae8b-43d2-9377-8a057e61543a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:16.296545 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645fc04-ae8b-43d2-9377-8a057e61543a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:16.313637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-2645fc04-ae8b-43d2-9377-8a057e61543a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:16.313637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:16.313637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:16.313637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:16.313637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:16.313637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:16.313637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:16.313637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:16.313637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:16.313637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:16.313637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:16.313637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:16.313637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:16.313637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:16.313637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:16.313637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:16.314668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:16.315577 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:16.315577 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:16.315577 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:16.315577 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 403/802] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:16.319089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645fc04-ae8b-43d2-9377-8a057e61543a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:16.389618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c5bc37-f3c1-4968-9997-bc577fabae03 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:16.428744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde505ad-0056-4ea7-bcef-67589bc0b3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:16.429380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde505ad-0056-4ea7-bcef-67589bc0b3cd None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:16.429660 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde505ad-0056-4ea7-bcef-67589bc0b3cd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:16.429933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde505ad-0056-4ea7-bcef-67589bc0b3cd None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:16.430690 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde505ad-0056-4ea7-bcef-67589bc0b3cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:16.430935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde505ad-0056-4ea7-bcef-67589bc0b3cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:16.432299 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde505ad-0056-4ea7-bcef-67589bc0b3cd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:16.432942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645fc04-ae8b-43d2-9377-8a057e61543a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:16.445540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 400/803] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:14:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:16.445540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365830c-56ac-41e2-93af-5111f99dcd8e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:16.445540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365830c-56ac-41e2-93af-5111f99dcd8e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:16.445540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6365830c-56ac-41e2-93af-5111f99dcd8e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:16.445540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 401/804] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:16.445540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365830c-56ac-41e2-93af-5111f99dcd8e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:16.476076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333b073-c43c-460b-9437-cd0cbc8593cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:16.476702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333b073-c43c-460b-9437-cd0cbc8593cc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:16.476978 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333b073-c43c-460b-9437-cd0cbc8593cc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:16.477247 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0333b073-c43c-460b-9437-cd0cbc8593cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:16.480294 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:16.484234 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333b073-c43c-460b-9437-cd0cbc8593cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:16.485631 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0333b073-c43c-460b-9437-cd0cbc8593cc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:16.497227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-0333b073-c43c-460b-9437-cd0cbc8593cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:16.497227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:16.497227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:16.497227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:16.497227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:16.497227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:16.497227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:16.497227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:16.497227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:16.497227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:16.497227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:16.497227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:16.497227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:16.497227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:16.497227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:16.498180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:16.499111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:16.499111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:16.499111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:16.499111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:16.499111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:16.499111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:16.499111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:16.499111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:16.499111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 404/805] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:16.499595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333b073-c43c-460b-9437-cd0cbc8593cc None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:16.554889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:16.557804 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6365830c-56ac-41e2-93af-5111f99dcd8e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:16.593046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77b3bc-42a2-486b-9903-4c9c862d6895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:16.593653 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77b3bc-42a2-486b-9903-4c9c862d6895 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:16.593930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77b3bc-42a2-486b-9903-4c9c862d6895 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:16.594202 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77b3bc-42a2-486b-9903-4c9c862d6895 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:16.594868 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77b3bc-42a2-486b-9903-4c9c862d6895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:16.595170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77b3bc-42a2-486b-9903-4c9c862d6895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:16.596534 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77b3bc-42a2-486b-9903-4c9c862d6895 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:16.608393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 402/806] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:16.609574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cbd27-7233-457c-9de4-bcd646a2c57d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:16.609852 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cbd27-7233-457c-9de4-bcd646a2c57d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:16.610127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cbd27-7233-457c-9de4-bcd646a2c57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:16.612717 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0333b073-c43c-460b-9437-cd0cbc8593cc None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:16.628073 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-921cbd27-7233-457c-9de4-bcd646a2c57d None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:16.629282 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-921cbd27-7233-457c-9de4-bcd646a2c57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:16.649757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bd2360-cfd9-451a-a194-89aa6ca54a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:16.650406 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd2360-cfd9-451a-a194-89aa6ca54a06 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:16.650680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd2360-cfd9-451a-a194-89aa6ca54a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:16.651199 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bd2360-cfd9-451a-a194-89aa6ca54a06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:16.661404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd2360-cfd9-451a-a194-89aa6ca54a06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:16.662882 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bd2360-cfd9-451a-a194-89aa6ca54a06 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:16.677257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-72bd2360-cfd9-451a-a194-89aa6ca54a06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:16.677257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:16.677257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:16.677257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:16.677257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:16.677257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:16.677257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:16.677257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:16.677257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:16.677257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:16.677257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:16.677257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:16.677257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:16.677257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:16.677257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:16.677257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:16.678298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:16.679232 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:16.679232 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:16.679232 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:16.679232 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 405/807] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:16.679878 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cded21-ea98-4157-b717-f7223a6efa62 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:16.679878 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cded21-ea98-4157-b717-f7223a6efa62 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:16.680228 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cded21-ea98-4157-b717-f7223a6efa62 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:16.681593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 406/808] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:16 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:16.682469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cded21-ea98-4157-b717-f7223a6efa62 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:16.699808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 403/809] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:16.700654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cbd27-7233-457c-9de4-bcd646a2c57d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:16.736762 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:16.795545 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:16.800813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cded21-ea98-4157-b717-f7223a6efa62 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:16.830284 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cbd27-7233-457c-9de4-bcd646a2c57d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:16.847858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a5337-e866-441e-be98-5706f4195c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:16.849058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a5337-e866-441e-be98-5706f4195c44 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:16.849340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a5337-e866-441e-be98-5706f4195c44 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:16.849613 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4a5337-e866-441e-be98-5706f4195c44 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:16.850341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a5337-e866-441e-be98-5706f4195c44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:16.850649 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a5337-e866-441e-be98-5706f4195c44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:16.852084 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4a5337-e866-441e-be98-5706f4195c44 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:16.865787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 407/810] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:16.866652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a5337-e866-441e-be98-5706f4195c44 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:16.875449 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d343ea-fcf9-4683-896e-2e035a3e108f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:16.876330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d343ea-fcf9-4683-896e-2e035a3e108f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:16.876757 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d343ea-fcf9-4683-896e-2e035a3e108f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:16.877296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d343ea-fcf9-4683-896e-2e035a3e108f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:16.885723 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d343ea-fcf9-4683-896e-2e035a3e108f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:16.887322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d343ea-fcf9-4683-896e-2e035a3e108f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:16.901972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-23d343ea-fcf9-4683-896e-2e035a3e108f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:16.901972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:16.901972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:16.901972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:16.901972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:16.901972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:16.901972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:16.901972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:16.901972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:16.901972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:16.901972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:16.901972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:16.901972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:16.901972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:16.901972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:16.902926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:16.903897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:16.903897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:16.903897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:16.903897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:16.903897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:16.903897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:16.903897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:16.903897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:16.903897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 404/811] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:16.904879 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c7bb9-5c34-4c13-8499-a500a072e192 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:16.904879 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c7bb9-5c34-4c13-8499-a500a072e192 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:16.905035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34c7bb9-5c34-4c13-8499-a500a072e192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:16.924224 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d34c7bb9-5c34-4c13-8499-a500a072e192 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:16.925554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d34c7bb9-5c34-4c13-8499-a500a072e192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:16.985113 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4a5337-e866-441e-be98-5706f4195c44 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:17.008181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 405/812] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:17.008922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c7bb9-5c34-4c13-8499-a500a072e192 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:17.023732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:17.031314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1ed7c-96fc-4c8c-9d3c-9038544f2710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:17.032025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1ed7c-96fc-4c8c-9d3c-9038544f2710 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:17.032306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1ed7c-96fc-4c8c-9d3c-9038544f2710 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:17.032575 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1ed7c-96fc-4c8c-9d3c-9038544f2710 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:17.033229 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1ed7c-96fc-4c8c-9d3c-9038544f2710 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:17.033530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1ed7c-96fc-4c8c-9d3c-9038544f2710 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:17.035159 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce1ed7c-96fc-4c8c-9d3c-9038544f2710 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:17.050260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 408/813] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:17.050260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1ed7c-96fc-4c8c-9d3c-9038544f2710 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:17.074766 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:17.130400 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34c7bb9-5c34-4c13-8499-a500a072e192 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:17.164942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1ed7c-96fc-4c8c-9d3c-9038544f2710 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:17.169221 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2d250-565a-420b-bbe9-e69971c0dfb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:17.170113 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2d250-565a-420b-bbe9-e69971c0dfb0 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:17.170596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2d250-565a-420b-bbe9-e69971c0dfb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:17.171018 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa2d250-565a-420b-bbe9-e69971c0dfb0 None admin] PATH_INFO: `/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/ad682eade4104a799b21f47aa7cdd4f5/roles/59a2b50693a646fc8332b835233486ce` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:17.187409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2d250-565a-420b-bbe9-e69971c0dfb0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c010096b77f4cd3b2f32086b1ba7ea7, user_id=ad682eade4104a799b21f47aa7cdd4f5, role_id=59a2b50693a646fc8332b835233486ce)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:17.189203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa2d250-565a-420b-bbe9-e69971c0dfb0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:17.199537 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546645ed-cbb2-4e66-875a-8c11354a1a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:17.200258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546645ed-cbb2-4e66-875a-8c11354a1a1e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:17.200391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546645ed-cbb2-4e66-875a-8c11354a1a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:17.200595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546645ed-cbb2-4e66-875a-8c11354a1a1e None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:17.201285 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546645ed-cbb2-4e66-875a-8c11354a1a1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:17.201588 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546645ed-cbb2-4e66-875a-8c11354a1a1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:17.203028 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546645ed-cbb2-4e66-875a-8c11354a1a1e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:17.205517 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 406/814] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:14:17 2022] PUT /identity/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/ad682eade4104a799b21f47aa7cdd4f5/roles/59a2b50693a646fc8332b835233486ce => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:14:17.207066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2d250-565a-420b-bbe9-e69971c0dfb0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:17.234391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 409/815] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:17.235643 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546645ed-cbb2-4e66-875a-8c11354a1a1e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:17.286322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:17.333156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa2d250-565a-420b-bbe9-e69971c0dfb0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:17.344082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:17.371933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546645ed-cbb2-4e66-875a-8c11354a1a1e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:17.373619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fab47-8022-41df-bea3-15b3c9e6e62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:17.374242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fab47-8022-41df-bea3-15b3c9e6e62e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:17.374545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fab47-8022-41df-bea3-15b3c9e6e62e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:17.374876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fab47-8022-41df-bea3-15b3c9e6e62e None admin] PATH_INFO: `/v3/services/network` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:17.375469 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fab47-8022-41df-bea3-15b3c9e6e62e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:17.376871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fab47-8022-41df-bea3-15b3c9e6e62e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6e7fab47-8022-41df-bea3-15b3c9e6e62e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:17.386626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:17.387667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:17.387667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:17.387667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:17.387667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:17.387667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:17.387667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:17.387667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:17.387667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:17.387667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:17.387667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:17.387667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:17.387667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:17.387667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:17.387667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:17.387667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:17.387667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:17.388483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:17.388483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:17.388483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:17.388483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:17.388483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:17.388483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:17.388483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:17.388483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:17.388483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:17.388483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:17.388483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:17.388483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:17.388483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 15:14:17.388483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:17.389528 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 407/816] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:17 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:17.389860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fab47-8022-41df-bea3-15b3c9e6e62e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:17.414119 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbc867-0dbd-4353-a222-b7e8d762ca9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:17.414787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbc867-0dbd-4353-a222-b7e8d762ca9e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:17.415064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbc867-0dbd-4353-a222-b7e8d762ca9e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:17.415353 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbc867-0dbd-4353-a222-b7e8d762ca9e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:17.426717 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbc867-0dbd-4353-a222-b7e8d762ca9e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:17.428850 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbc867-0dbd-4353-a222-b7e8d762ca9e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:17.446598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-69fbc867-0dbd-4353-a222-b7e8d762ca9e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:14:17.446598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:17.446598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:17.446598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:17.446598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:17.446598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:17.446598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:17.446598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:17.446598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:17.446598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:17.446598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:17.446598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:17.446598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:17.446598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:17.446598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:17.446598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:17.448001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:17.448950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:17.448950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:17.448950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:17.448950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:17.448950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:17.448950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:17.448950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:17.448950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:17.448950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:14:17.448950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:17.448950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 410/817] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 15:14:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:17.448950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbc867-0dbd-4353-a222-b7e8d762ca9e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:17.514597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fab47-8022-41df-bea3-15b3c9e6e62e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:17.548143 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:17.559197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836320b5-74af-458a-8a0e-ddd98595d926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:17.559864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836320b5-74af-458a-8a0e-ddd98595d926 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:17.560147 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836320b5-74af-458a-8a0e-ddd98595d926 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:17.560427 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836320b5-74af-458a-8a0e-ddd98595d926 None admin] PATH_INFO: `/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/0679072b37db41e880a95b738bea47ea/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:17.562420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836320b5-74af-458a-8a0e-ddd98595d926 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c010096b77f4cd3b2f32086b1ba7ea7, user_id=0679072b37db41e880a95b738bea47ea, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:17.562528 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbc867-0dbd-4353-a222-b7e8d762ca9e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:17.563854 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836320b5-74af-458a-8a0e-ddd98595d926 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:17.586264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 408/818] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:14:17 2022] PUT /identity/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/0679072b37db41e880a95b738bea47ea/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:14:17.586264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836320b5-74af-458a-8a0e-ddd98595d926 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:17.611383 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903fc13b-327d-4f7c-9227-b7f0eed9ed67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:17.612318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fc13b-327d-4f7c-9227-b7f0eed9ed67 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:17.612594 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fc13b-327d-4f7c-9227-b7f0eed9ed67 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:17.612858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903fc13b-327d-4f7c-9227-b7f0eed9ed67 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:17.613952 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903fc13b-327d-4f7c-9227-b7f0eed9ed67 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:17.616214 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903fc13b-327d-4f7c-9227-b7f0eed9ed67 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-903fc13b-327d-4f7c-9227-b7f0eed9ed67 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:17.627240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:17.628364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:17.628364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:17.628364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:17.628364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:17.628364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:17.628364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:17.628364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:17.628364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:17.628364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:17.628364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:17.628364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:17.628364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:17.628364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:17.628364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:17.628364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:17.628364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:17.629190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:17.629190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:17.629190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:17.629190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:17.629190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:17.629190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:17.629190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:17.629190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:17.629190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:17.629190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:17.629190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:17.629190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:17.629190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 15:14:17.629190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:17.629190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 411/819] 10.222.0.76 () {56 vars in 1129 bytes} [Wed Aug 24 15:14:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:17.630511 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903fc13b-327d-4f7c-9227-b7f0eed9ed67 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:17.646449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:17.710904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836320b5-74af-458a-8a0e-ddd98595d926 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:17.756193 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f576a18-f8dd-4eed-9665-34913093411c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:17.756810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f576a18-f8dd-4eed-9665-34913093411c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:17.757090 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f576a18-f8dd-4eed-9665-34913093411c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:17.757358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f576a18-f8dd-4eed-9665-34913093411c None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:17.758887 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f576a18-f8dd-4eed-9665-34913093411c None admin] RBAC: Adding query filter params (name=network) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:17.759211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f576a18-f8dd-4eed-9665-34913093411c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:17.760601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f576a18-f8dd-4eed-9665-34913093411c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:17.765814 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 409/820] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 15:14:17 2022] GET /identity/v3/services?name=network => generated 124 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:17.766465 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f576a18-f8dd-4eed-9665-34913093411c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:17.773613 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903fc13b-327d-4f7c-9227-b7f0eed9ed67 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:17.814731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6b4ba-cf04-470b-a724-0c6b51a62ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:17.815317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6b4ba-cf04-470b-a724-0c6b51a62ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:17.815698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6b4ba-cf04-470b-a724-0c6b51a62ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:17.815848 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb6b4ba-cf04-470b-a724-0c6b51a62ae7 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:17.816643 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6b4ba-cf04-470b-a724-0c6b51a62ae7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:17.816977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6b4ba-cf04-470b-a724-0c6b51a62ae7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:17.818569 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb6b4ba-cf04-470b-a724-0c6b51a62ae7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:17.834933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:17.841984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 412/821] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 15:14:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:17.841984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6b4ba-cf04-470b-a724-0c6b51a62ae7 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:17.909257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:17.915352 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f576a18-f8dd-4eed-9665-34913093411c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:17.956363 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb6b4ba-cf04-470b-a724-0c6b51a62ae7 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:17.965217 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e5906-06dd-4d8e-ba7c-165c4fba344c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:17.966099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e5906-06dd-4d8e-ba7c-165c4fba344c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:17.966667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e5906-06dd-4d8e-ba7c-165c4fba344c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:17.967687 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286e5906-06dd-4d8e-ba7c-165c4fba344c None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:17.968959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e5906-06dd-4d8e-ba7c-165c4fba344c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:17.969774 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e5906-06dd-4d8e-ba7c-165c4fba344c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:17.971679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286e5906-06dd-4d8e-ba7c-165c4fba344c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:17.979914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 410/822] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 24 15:14:17 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:17.981326 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e5906-06dd-4d8e-ba7c-165c4fba344c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:17.999979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d7cb5-8c29-4a63-a1e9-cf7ca2f561bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:18.001160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d7cb5-8c29-4a63-a1e9-cf7ca2f561bb None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:18.002127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d7cb5-8c29-4a63-a1e9-cf7ca2f561bb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:18.002650 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2d7cb5-8c29-4a63-a1e9-cf7ca2f561bb None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:18.003518 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d7cb5-8c29-4a63-a1e9-cf7ca2f561bb None admin] RBAC: Adding query filter params (type=network) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:18.004301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d7cb5-8c29-4a63-a1e9-cf7ca2f561bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:18.006559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2d7cb5-8c29-4a63-a1e9-cf7ca2f561bb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:18.018147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 413/823] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 15:14:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:18.019674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d7cb5-8c29-4a63-a1e9-cf7ca2f561bb None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:18.101883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286e5906-06dd-4d8e-ba7c-165c4fba344c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:18.116637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:18.142629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030dde5f-b127-4285-92f5-bb331bd4ff88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:18.143386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dde5f-b127-4285-92f5-bb331bd4ff88 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:18.143732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dde5f-b127-4285-92f5-bb331bd4ff88 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:18.144097 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dde5f-b127-4285-92f5-bb331bd4ff88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:18.164004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2d7cb5-8c29-4a63-a1e9-cf7ca2f561bb None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:18.206180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:18.211507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b443a74-0a8a-4cc0-ad06-04c3ae44b2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:18.212125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b443a74-0a8a-4cc0-ad06-04c3ae44b2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:18.212411 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b443a74-0a8a-4cc0-ad06-04c3ae44b2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:18.212785 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b443a74-0a8a-4cc0-ad06-04c3ae44b2a7 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:18.213705 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b443a74-0a8a-4cc0-ad06-04c3ae44b2a7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:18.214065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b443a74-0a8a-4cc0-ad06-04c3ae44b2a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:18.215551 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b443a74-0a8a-4cc0-ad06-04c3ae44b2a7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:18.224649 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030dde5f-b127-4285-92f5-bb331bd4ff88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:18.226338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030dde5f-b127-4285-92f5-bb331bd4ff88 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:18.226494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 414/824] 10.222.0.76 () {56 vars in 1124 bytes} [Wed Aug 24 15:14:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:18.227371 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b443a74-0a8a-4cc0-ad06-04c3ae44b2a7 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:18.336813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 411/825] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:14:17 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:14:18.337741 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030dde5f-b127-4285-92f5-bb331bd4ff88 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:18.344426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b443a74-0a8a-4cc0-ad06-04c3ae44b2a7 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:18.354256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:18.386257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f514e57-be08-4d46-87bf-829b7a564ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:18.387118 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f514e57-be08-4d46-87bf-829b7a564ba1 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:18.387555 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f514e57-be08-4d46-87bf-829b7a564ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:18.387973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f514e57-be08-4d46-87bf-829b7a564ba1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:18.388879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f514e57-be08-4d46-87bf-829b7a564ba1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:18.390482 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f514e57-be08-4d46-87bf-829b7a564ba1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:18.416192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 415/826] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:14:18 2022] POST /identity/v3/endpoints => generated 337 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:14:18.417314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f514e57-be08-4d46-87bf-829b7a564ba1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:18.450369 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030dde5f-b127-4285-92f5-bb331bd4ff88 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:18.476595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:18.498309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90d089-8604-4e93-accd-f553eec5075d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:18.498309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90d089-8604-4e93-accd-f553eec5075d None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:18.498309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90d089-8604-4e93-accd-f553eec5075d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:18.498309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e90d089-8604-4e93-accd-f553eec5075d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:18.498758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e90d089-8604-4e93-accd-f553eec5075d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:18.500177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e90d089-8604-4e93-accd-f553eec5075d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:18.540069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 412/827] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:14:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:14:18.573604 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f514e57-be08-4d46-87bf-829b7a564ba1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:18.621032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e877e4-a9eb-46c1-805d-b0d33f21f3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:18.621767 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e877e4-a9eb-46c1-805d-b0d33f21f3dd None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:18.622150 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e877e4-a9eb-46c1-805d-b0d33f21f3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:18.622582 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e877e4-a9eb-46c1-805d-b0d33f21f3dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:18.639050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e877e4-a9eb-46c1-805d-b0d33f21f3dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:18.640544 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e877e4-a9eb-46c1-805d-b0d33f21f3dd None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:18.657930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-e1e877e4-a9eb-46c1-805d-b0d33f21f3dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:18.657930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:18.657930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:18.657930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:18.657930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:18.657930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:18.657930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:18.657930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:18.657930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:18.657930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:18.657930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:18.657930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:18.657930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:18.657930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:18.657930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:18.657930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:18.658975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:18.659912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:18.659912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:18.659912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:18.659912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 416/828] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:18.667503 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90d089-8604-4e93-accd-f553eec5075d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:18.765733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:18.818162 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e90d089-8604-4e93-accd-f553eec5075d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:18.862104 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5f5ef-378b-490e-b301-e3bc10b0a1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:18.862771 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5f5ef-378b-490e-b301-e3bc10b0a1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:18.863056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5f5ef-378b-490e-b301-e3bc10b0a1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:18.863333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e5f5ef-378b-490e-b301-e3bc10b0a1b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:18.864016 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5f5ef-378b-490e-b301-e3bc10b0a1b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:18.864320 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5f5ef-378b-490e-b301-e3bc10b0a1b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:18.865698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e5f5ef-378b-490e-b301-e3bc10b0a1b5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:18.885583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 413/829] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:18.887205 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e877e4-a9eb-46c1-805d-b0d33f21f3dd None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:19.016960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:19.027665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e877e4-a9eb-46c1-805d-b0d33f21f3dd None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:19.063584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dbd385-4557-46f6-8457-5d253e094893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:19.064214 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbd385-4557-46f6-8457-5d253e094893 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:19.064490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbd385-4557-46f6-8457-5d253e094893 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:19.064764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dbd385-4557-46f6-8457-5d253e094893 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:19.072564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbd385-4557-46f6-8457-5d253e094893 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:19.074031 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dbd385-4557-46f6-8457-5d253e094893 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:19.085813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-20dbd385-4557-46f6-8457-5d253e094893 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:19.085813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:19.085813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:19.085813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:19.085813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:19.085813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:19.085813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:19.085813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:19.085813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:19.085813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:19.085813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:19.085813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:19.085813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:19.085813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:19.085813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:19.086770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:19.088161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:19.088161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:19.088161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:19.088161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:19.088161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:19.088161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:19.088161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:19.088161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:19.088161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 417/830] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:19.095054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5f5ef-378b-490e-b301-e3bc10b0a1b5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:19.190968 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35d4b1-86f7-485b-b8c5-ae4a3e823afd None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:19.191306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35d4b1-86f7-485b-b8c5-ae4a3e823afd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:19.191522 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c35d4b1-86f7-485b-b8c5-ae4a3e823afd None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:19.193163 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 418/831] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:19.198451 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac76848-153a-4d76-a335-8076a121f10d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:19.198735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac76848-153a-4d76-a335-8076a121f10d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:19.199009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac76848-153a-4d76-a335-8076a121f10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:19.203814 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e5f5ef-378b-490e-b301-e3bc10b0a1b5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:19.217329 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8ac76848-153a-4d76-a335-8076a121f10d None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:19.218719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8ac76848-153a-4d76-a335-8076a121f10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:19.233547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:19.242929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010b1c6-35f7-4f89-a7bb-9b3e7565d95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:19.243666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010b1c6-35f7-4f89-a7bb-9b3e7565d95e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:19.244059 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010b1c6-35f7-4f89-a7bb-9b3e7565d95e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:19.244405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d010b1c6-35f7-4f89-a7bb-9b3e7565d95e None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:19.245292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010b1c6-35f7-4f89-a7bb-9b3e7565d95e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:19.245603 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010b1c6-35f7-4f89-a7bb-9b3e7565d95e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:19.247257 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d010b1c6-35f7-4f89-a7bb-9b3e7565d95e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:19.265742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 414/832] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:19.267055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010b1c6-35f7-4f89-a7bb-9b3e7565d95e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:19.298354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 419/833] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:19.375275 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d010b1c6-35f7-4f89-a7bb-9b3e7565d95e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:19.411669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec497d57-5364-4b83-8607-9b6c350f4788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:19.412280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec497d57-5364-4b83-8607-9b6c350f4788 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:19.412562 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec497d57-5364-4b83-8607-9b6c350f4788 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:19.412839 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec497d57-5364-4b83-8607-9b6c350f4788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:19.413566 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec497d57-5364-4b83-8607-9b6c350f4788 None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=8a5fd31a2db944f596912e4f0aa26835) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:19.413875 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec497d57-5364-4b83-8607-9b6c350f4788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:19.415321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec497d57-5364-4b83-8607-9b6c350f4788 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:19.426092 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 415/834] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:14:19 2022] GET /identity/v3/role_assignments?user.id=8a5fd31a2db944f596912e4f0aa26835&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:19.565179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac76848-153a-4d76-a335-8076a121f10d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:19.652044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:19.671018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac76848-153a-4d76-a335-8076a121f10d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:19.708409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e489df4-2872-49b9-b35c-16851093713a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:19.709042 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e489df4-2872-49b9-b35c-16851093713a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:19.709319 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e489df4-2872-49b9-b35c-16851093713a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:19.709587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e489df4-2872-49b9-b35c-16851093713a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:19.720656 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e489df4-2872-49b9-b35c-16851093713a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:19.725559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-0e489df4-2872-49b9-b35c-16851093713a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:14:19.725559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:19.725559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:19.725559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:19.725559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:19.725559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:19.725559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:19.725559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:19.725559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:19.725559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:19.725559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:19.725559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:19.725559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:19.725559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:19.725559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:19.725559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:19.726542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:19.726542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:19.726542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:19.726542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:19.726542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:19.726542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:19.726542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:19.726542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:19.726542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:19.726542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:19.726542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:19.726542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:19.726542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:19.726542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:19.726542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:19.726542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:19.727386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:19.727386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:19.727386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:19.727386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:19.727386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:19.727386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:19.727386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:19.727386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:19.727386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:19.727386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:19.727386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:14:19.727386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:19.730355 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 420/835] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:14:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:19.736604 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec497d57-5364-4b83-8607-9b6c350f4788 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:19.793668 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:19.798925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d194b-00b3-41a5-ada9-a96dc58a002a None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:19.799283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d194b-00b3-41a5-ada9-a96dc58a002a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:19.799475 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d194b-00b3-41a5-ada9-a96dc58a002a None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:19.802058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 421/836] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:19.807585 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd397a0-486a-41ec-b2a5-4e159e69aac8 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:19.807873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd397a0-486a-41ec-b2a5-4e159e69aac8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:19.808144 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd397a0-486a-41ec-b2a5-4e159e69aac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:19.828675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8fd397a0-486a-41ec-b2a5-4e159e69aac8 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:19.829922 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8fd397a0-486a-41ec-b2a5-4e159e69aac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:19.852266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec497d57-5364-4b83-8607-9b6c350f4788 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:19.900029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0e2f7-b819-4776-9700-e71220018e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:19.900749 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0e2f7-b819-4776-9700-e71220018e61 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:19.901291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0e2f7-b819-4776-9700-e71220018e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:19.901562 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be0e2f7-b819-4776-9700-e71220018e61 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:19.902244 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0e2f7-b819-4776-9700-e71220018e61 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:19.902557 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0e2f7-b819-4776-9700-e71220018e61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:19.903926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be0e2f7-b819-4776-9700-e71220018e61 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:19.905748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 422/837] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:19.922486 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 416/838] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:14:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:19.923921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd397a0-486a-41ec-b2a5-4e159e69aac8 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:20.000612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:20.034047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd397a0-486a-41ec-b2a5-4e159e69aac8 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:20.072509 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403dce2-6871-48b1-aa8a-df1e3f045972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:20.073098 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403dce2-6871-48b1-aa8a-df1e3f045972 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:20.073451 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403dce2-6871-48b1-aa8a-df1e3f045972 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:20.073641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403dce2-6871-48b1-aa8a-df1e3f045972 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:20.085586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403dce2-6871-48b1-aa8a-df1e3f045972 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:20.087143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4403dce2-6871-48b1-aa8a-df1e3f045972 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:20.103874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-4403dce2-6871-48b1-aa8a-df1e3f045972 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:20.103874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:20.103874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:20.103874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:20.103874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:20.103874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:20.103874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:20.103874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:20.103874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:20.103874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:20.103874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:20.103874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:20.103874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:20.103874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:20.103874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:20.103874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:20.104856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:20.105735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:20.105735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:20.105735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:20.108041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 423/839] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:20.114597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0e2f7-b819-4776-9700-e71220018e61 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:20.209503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403dce2-6871-48b1-aa8a-df1e3f045972 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:20.239924 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be0e2f7-b819-4776-9700-e71220018e61 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:20.253589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:20.280907 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554474f-f323-4234-bfc4-f498e287cccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:20.281597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554474f-f323-4234-bfc4-f498e287cccc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:20.281878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554474f-f323-4234-bfc4-f498e287cccc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:20.282150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b554474f-f323-4234-bfc4-f498e287cccc None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:20.283033 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554474f-f323-4234-bfc4-f498e287cccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:20.283157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554474f-f323-4234-bfc4-f498e287cccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:20.284543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b554474f-f323-4234-bfc4-f498e287cccc None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:20.303963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 417/840] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:20.304834 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554474f-f323-4234-bfc4-f498e287cccc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:20.347584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403dce2-6871-48b1-aa8a-df1e3f045972 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:20.375892 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:20.388236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03773c-4ce7-43a0-9df6-80f8f1153b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:20.388876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03773c-4ce7-43a0-9df6-80f8f1153b83 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:20.389150 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03773c-4ce7-43a0-9df6-80f8f1153b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:20.389420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c03773c-4ce7-43a0-9df6-80f8f1153b83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:20.400852 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c03773c-4ce7-43a0-9df6-80f8f1153b83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:20.406444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-4c03773c-4ce7-43a0-9df6-80f8f1153b83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:20.406444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:20.406444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:20.406444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:20.406444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:20.406444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:20.406444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:20.406444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:20.406444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:20.406444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:20.406444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:20.406444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:20.406444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:20.406444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:20.406444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:20.406444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:20.407364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:20.407364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:20.407364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:20.407364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:20.407364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:20.407364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:20.407364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:20.407364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:20.407364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:20.407364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:20.407364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:20.407364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:20.407364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:20.407364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:20.407364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:20.407364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:20.408185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:20.408185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:20.408185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:20.408185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:20.408185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:20.408185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:20.408185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:20.408185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:20.408185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:20.408185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:20.408185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:20.408185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:20.408185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 424/841] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:14:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:20.413723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03773c-4ce7-43a0-9df6-80f8f1153b83 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:20.444484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b554474f-f323-4234-bfc4-f498e287cccc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:20.484571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c75e89-33ca-4594-bece-e77878d4b0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:20.485504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c75e89-33ca-4594-bece-e77878d4b0f6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:20.486082 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c75e89-33ca-4594-bece-e77878d4b0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:20.486551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c75e89-33ca-4594-bece-e77878d4b0f6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:20.494750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c75e89-33ca-4594-bece-e77878d4b0f6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:20.496325 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c75e89-33ca-4594-bece-e77878d4b0f6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:20.522353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f7c75e89-33ca-4594-bece-e77878d4b0f6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:14:20.522353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:20.522353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:20.522353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:20.522353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:20.522353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:20.522353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:20.522353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:20.522353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:20.522353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:20.522353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:20.522353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:20.522353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:20.522353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:20.522353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:20.522353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:20.523322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:20.524236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:20.524236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:20.524236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:20.524236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:20.524236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:20.524236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:20.524236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:20.524236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:20.524236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:14:20.524236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:20.532187 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 418/842] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 15:14:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:20.533229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c75e89-33ca-4594-bece-e77878d4b0f6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:20.552958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c03773c-4ce7-43a0-9df6-80f8f1153b83 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:20.586415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:20.595470 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01090d50-2b74-4b4b-b34a-f4b81d500993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:20.597094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01090d50-2b74-4b4b-b34a-f4b81d500993 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:20.597639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01090d50-2b74-4b4b-b34a-f4b81d500993 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:20.598886 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01090d50-2b74-4b4b-b34a-f4b81d500993 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:20.600262 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01090d50-2b74-4b4b-b34a-f4b81d500993 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:20.600981 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01090d50-2b74-4b4b-b34a-f4b81d500993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:20.603167 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01090d50-2b74-4b4b-b34a-f4b81d500993 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:20.615807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 425/843] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:14:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:20.621120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01090d50-2b74-4b4b-b34a-f4b81d500993 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:20.673294 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c75e89-33ca-4594-bece-e77878d4b0f6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:20.682186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:20.710369 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc0332c-90d8-4742-a124-9df8648535d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:20.710969 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0332c-90d8-4742-a124-9df8648535d3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:20.711246 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0332c-90d8-4742-a124-9df8648535d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:20.711541 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc0332c-90d8-4742-a124-9df8648535d3 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:20.712236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc0332c-90d8-4742-a124-9df8648535d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:20.712539 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc0332c-90d8-4742-a124-9df8648535d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:20.713957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc0332c-90d8-4742-a124-9df8648535d3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:20.732473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 419/844] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 24 15:14:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:20.733700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd16a6a-e3b6-4565-8660-2acb0fceeb9d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:20.733983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd16a6a-e3b6-4565-8660-2acb0fceeb9d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:20.734270 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd16a6a-e3b6-4565-8660-2acb0fceeb9d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:20.735584 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 420/845] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:20.740269 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd16a6a-e3b6-4565-8660-2acb0fceeb9d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:20.747145 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01090d50-2b74-4b4b-b34a-f4b81d500993 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:20.786403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f85c72-e02c-44cb-8907-dbde726223d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:20.787019 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f85c72-e02c-44cb-8907-dbde726223d8 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:20.787296 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f85c72-e02c-44cb-8907-dbde726223d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:20.787563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f85c72-e02c-44cb-8907-dbde726223d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:20.798735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f85c72-e02c-44cb-8907-dbde726223d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:20.800163 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f85c72-e02c-44cb-8907-dbde726223d8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:20.814396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-48f85c72-e02c-44cb-8907-dbde726223d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:20.814396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:20.814396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:20.814396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:20.814396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:20.814396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:20.814396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:20.814396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:20.814396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:20.814396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:20.814396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:20.814396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:20.814396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:20.814396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:20.814396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:20.814396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:20.815340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:20.816199 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:20.816199 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:20.816199 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:20.820781 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 426/846] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:20.822006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b2ce5-69eb-4b3c-9cf3-557a0805e833 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:20.822317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b2ce5-69eb-4b3c-9cf3-557a0805e833 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:20.822592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351b2ce5-69eb-4b3c-9cf3-557a0805e833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:20.845111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-351b2ce5-69eb-4b3c-9cf3-557a0805e833 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:20.846383 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-351b2ce5-69eb-4b3c-9cf3-557a0805e833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:20.846714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:20.863311 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd16a6a-e3b6-4565-8660-2acb0fceeb9d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:20.905207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07029d09-60bd-421b-a1ce-2fbcacd708be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:20.906110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07029d09-60bd-421b-a1ce-2fbcacd708be None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:20.906700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07029d09-60bd-421b-a1ce-2fbcacd708be None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:20.906973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07029d09-60bd-421b-a1ce-2fbcacd708be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:20.926256 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 427/847] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:20.926256 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b2ce5-69eb-4b3c-9cf3-557a0805e833 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:20.956532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:20.994254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07029d09-60bd-421b-a1ce-2fbcacd708be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:20.996446 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07029d09-60bd-421b-a1ce-2fbcacd708be None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:21.108142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:21.122734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 421/848] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:14:20 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:14:21.128493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07029d09-60bd-421b-a1ce-2fbcacd708be None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:21.165865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351b2ce5-69eb-4b3c-9cf3-557a0805e833 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:21.204142 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3167f-fac4-4b78-b3d7-291d5eba8613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:21.204752 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3167f-fac4-4b78-b3d7-291d5eba8613 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:21.205029 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3167f-fac4-4b78-b3d7-291d5eba8613 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:21.205306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3167f-fac4-4b78-b3d7-291d5eba8613 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:21.206055 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3167f-fac4-4b78-b3d7-291d5eba8613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:21.206496 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3167f-fac4-4b78-b3d7-291d5eba8613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:21.207926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3167f-fac4-4b78-b3d7-291d5eba8613 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:21.226668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 428/849] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:21.228836 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3167f-fac4-4b78-b3d7-291d5eba8613 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:21.250636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07029d09-60bd-421b-a1ce-2fbcacd708be None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:21.286820 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dbf72-b4d2-4a24-881e-f5eb1da2c6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:21.287453 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dbf72-b4d2-4a24-881e-f5eb1da2c6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:21.288064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dbf72-b4d2-4a24-881e-f5eb1da2c6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:21.288064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39dbf72-b4d2-4a24-881e-f5eb1da2c6c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:21.298497 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39dbf72-b4d2-4a24-881e-f5eb1da2c6c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:21.299933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39dbf72-b4d2-4a24-881e-f5eb1da2c6c6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:21.315276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a39dbf72-b4d2-4a24-881e-f5eb1da2c6c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:21.315276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:21.315276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:21.315276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:21.315276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:21.315276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:21.315276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:21.315276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:21.315276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:21.315276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:21.315276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:21.315276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:21.315276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:21.315276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:21.315276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:21.315276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:21.316423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:21.317911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:21.317911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:21.317911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:21.317911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 422/850] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:21.317911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dbf72-b4d2-4a24-881e-f5eb1da2c6c6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:21.334535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:21.354542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3167f-fac4-4b78-b3d7-291d5eba8613 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:21.364522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:21.403984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a4d9f-346b-4f02-9c24-b3fe901f2266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:21.404546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a4d9f-346b-4f02-9c24-b3fe901f2266 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:21.404824 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a4d9f-346b-4f02-9c24-b3fe901f2266 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:21.405094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675a4d9f-346b-4f02-9c24-b3fe901f2266 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:21.413845 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a4d9f-346b-4f02-9c24-b3fe901f2266 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:21.415351 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675a4d9f-346b-4f02-9c24-b3fe901f2266 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:21.427303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-675a4d9f-346b-4f02-9c24-b3fe901f2266 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:14:21.427303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:21.427303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:21.427303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:21.427303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:21.427303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:21.427303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:21.427303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:21.427303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:21.427303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:21.427303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:21.427303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:21.427303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:21.427303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:21.427303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:21.427303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:21.428286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:21.429335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:21.429335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:21.429335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:21.429335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:21.429335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:21.429335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:21.429335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:21.429335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:21.429335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 15:14:21.429335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:21.429335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 429/851] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:14:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:21.429932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a4d9f-346b-4f02-9c24-b3fe901f2266 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:21.443839 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39dbf72-b4d2-4a24-881e-f5eb1da2c6c6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:21.481261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6d7d9-65e8-42ea-9ccd-640218540ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:21.481997 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6d7d9-65e8-42ea-9ccd-640218540ffb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:21.482372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6d7d9-65e8-42ea-9ccd-640218540ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:21.482727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce6d7d9-65e8-42ea-9ccd-640218540ffb None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:21.483383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce6d7d9-65e8-42ea-9ccd-640218540ffb None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:21.484797 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce6d7d9-65e8-42ea-9ccd-640218540ffb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-cce6d7d9-65e8-42ea-9ccd-640218540ffb None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:21.494543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:21.495571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:21.495571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:21.495571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:21.495571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:21.495571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:21.495571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:21.495571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:21.495571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:21.495571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:21.495571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:21.495571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:21.495571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:21.495571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:21.495571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:21.495571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:21.495571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:21.496396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:21.496396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:21.496396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:21.496396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:21.496396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:21.496396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:21.496396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:21.496396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:21.496396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:21.496396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:21.496396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:21.496396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:21.496396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:14:21.496396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:21.496396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 423/852] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:14:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:21.497177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6d7d9-65e8-42ea-9ccd-640218540ffb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:21.548685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675a4d9f-346b-4f02-9c24-b3fe901f2266 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:21.583494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3dfb4-5daf-4020-8d95-b144eb557c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:21.584093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3dfb4-5daf-4020-8d95-b144eb557c0f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:21.584370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3dfb4-5daf-4020-8d95-b144eb557c0f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:21.584641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3dfb4-5daf-4020-8d95-b144eb557c0f None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:21.585279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3dfb4-5daf-4020-8d95-b144eb557c0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:21.585580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3dfb4-5daf-4020-8d95-b144eb557c0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:21.586977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb3dfb4-5daf-4020-8d95-b144eb557c0f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:21.599391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 430/853] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:21.607565 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3dfb4-5daf-4020-8d95-b144eb557c0f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:21.611183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce6d7d9-65e8-42ea-9ccd-640218540ffb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:21.616886 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:21.619851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:21.650609 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75fa788-2c39-4b8b-9d4c-6a98efa69761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:21.651464 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fa788-2c39-4b8b-9d4c-6a98efa69761 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:21.651738 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fa788-2c39-4b8b-9d4c-6a98efa69761 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:21.652003 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75fa788-2c39-4b8b-9d4c-6a98efa69761 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:21.652730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fa788-2c39-4b8b-9d4c-6a98efa69761 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:21.653124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fa788-2c39-4b8b-9d4c-6a98efa69761 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:21.654722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75fa788-2c39-4b8b-9d4c-6a98efa69761 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:21.680802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 424/854] 10.222.0.76 () {56 vars in 1135 bytes} [Wed Aug 24 15:14:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:21.680802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788524d-ef44-4ad5-bff2-1704741375da None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:21.680802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788524d-ef44-4ad5-bff2-1704741375da None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:21.680802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788524d-ef44-4ad5-bff2-1704741375da None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:21.680802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 425/855] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:21.680802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788524d-ef44-4ad5-bff2-1704741375da None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:21.712761 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb3dfb4-5daf-4020-8d95-b144eb557c0f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:21.747431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764153df-0e96-41b9-96e0-0276b5cff1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:21.748038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764153df-0e96-41b9-96e0-0276b5cff1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:21.748314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764153df-0e96-41b9-96e0-0276b5cff1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:21.748590 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764153df-0e96-41b9-96e0-0276b5cff1c4 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:21.749211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764153df-0e96-41b9-96e0-0276b5cff1c4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:21.749604 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764153df-0e96-41b9-96e0-0276b5cff1c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:21.751003 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764153df-0e96-41b9-96e0-0276b5cff1c4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:21.755639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 431/856] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:14:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:21.756424 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764153df-0e96-41b9-96e0-0276b5cff1c4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:21.791767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788524d-ef44-4ad5-bff2-1704741375da None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:21.825020 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a2c449-e7b2-4abc-90f5-a16184784925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:21.825642 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2c449-e7b2-4abc-90f5-a16184784925 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:21.825915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2c449-e7b2-4abc-90f5-a16184784925 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:21.826180 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a2c449-e7b2-4abc-90f5-a16184784925 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:21.828175 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:21.834102 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2c449-e7b2-4abc-90f5-a16184784925 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:21.835586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a2c449-e7b2-4abc-90f5-a16184784925 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:21.851165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c5a2c449-e7b2-4abc-90f5-a16184784925 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:21.851165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:21.851165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:21.851165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:21.851165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:21.851165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:21.851165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:21.851165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:21.851165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:21.851165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:21.851165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:21.851165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:21.851165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:21.851165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:21.851165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:21.852079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:21.853736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:21.853736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:21.853736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:21.853736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:21.853736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:21.853736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:21.853736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:21.853736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:21.853736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 426/857] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:21.854256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5534d2-7d47-4aea-aeb6-cdbdfe00aca5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:21.854256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5534d2-7d47-4aea-aeb6-cdbdfe00aca5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:21.854514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5534d2-7d47-4aea-aeb6-cdbdfe00aca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:21.868277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764153df-0e96-41b9-96e0-0276b5cff1c4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:21.871885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4a5534d2-7d47-4aea-aeb6-cdbdfe00aca5 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:21.873095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4a5534d2-7d47-4aea-aeb6-cdbdfe00aca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:21.890413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:21.904474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09499b-c8da-4341-8b53-531f3db5e5a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:21.905125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09499b-c8da-4341-8b53-531f3db5e5a0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:21.905347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09499b-c8da-4341-8b53-531f3db5e5a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:21.905617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a09499b-c8da-4341-8b53-531f3db5e5a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:21.943987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 427/858] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:21 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:21.945071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5534d2-7d47-4aea-aeb6-cdbdfe00aca5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:21.983160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a09499b-c8da-4341-8b53-531f3db5e5a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:21.984843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a09499b-c8da-4341-8b53-531f3db5e5a0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:22.062540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5534d2-7d47-4aea-aeb6-cdbdfe00aca5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:22.074255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:22.100793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e4f24-7fe9-4a5d-b78d-a13c4a6d36d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:22.101450 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e4f24-7fe9-4a5d-b78d-a13c4a6d36d5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:22.101775 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e4f24-7fe9-4a5d-b78d-a13c4a6d36d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:22.102028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9e4f24-7fe9-4a5d-b78d-a13c4a6d36d5 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:22.102708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e4f24-7fe9-4a5d-b78d-a13c4a6d36d5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:22.103012 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e4f24-7fe9-4a5d-b78d-a13c4a6d36d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:22.104409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9e4f24-7fe9-4a5d-b78d-a13c4a6d36d5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:22.109465 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 428/859] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:14:21 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:22.114261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e4f24-7fe9-4a5d-b78d-a13c4a6d36d5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:22.115422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 432/860] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:14:21 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:14:22.116783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09499b-c8da-4341-8b53-531f3db5e5a0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:22.145327 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:22.231131 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a09499b-c8da-4341-8b53-531f3db5e5a0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:22.253655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9e4f24-7fe9-4a5d-b78d-a13c4a6d36d5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:22.272292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55991607-f23a-4388-bb0d-2df9366bf2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:22.272914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55991607-f23a-4388-bb0d-2df9366bf2e2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:22.273183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55991607-f23a-4388-bb0d-2df9366bf2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:22.273449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55991607-f23a-4388-bb0d-2df9366bf2e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:22.284489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55991607-f23a-4388-bb0d-2df9366bf2e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:22.286505 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55991607-f23a-4388-bb0d-2df9366bf2e2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:22.297230 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cce5b6-b6a2-4f68-9e15-64d0a0e1919c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:22.297841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cce5b6-b6a2-4f68-9e15-64d0a0e1919c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:22.298119 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cce5b6-b6a2-4f68-9e15-64d0a0e1919c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:22.298408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cce5b6-b6a2-4f68-9e15-64d0a0e1919c None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:22.299108 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cce5b6-b6a2-4f68-9e15-64d0a0e1919c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:22.299408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cce5b6-b6a2-4f68-9e15-64d0a0e1919c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:22.300830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cce5b6-b6a2-4f68-9e15-64d0a0e1919c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:22.305433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-55991607-f23a-4388-bb0d-2df9366bf2e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:22.305433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:22.305433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:22.305433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:22.305433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:22.305433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:22.305433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:22.305433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:22.305433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:22.305433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:22.305433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:22.305433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:22.305433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:22.305433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:22.305433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:22.305433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:22.306507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:22.307373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:22.307373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:22.307373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:22.307940 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 433/861] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:22.308588 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55991607-f23a-4388-bb0d-2df9366bf2e2 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:22.313948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 429/862] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:22.315033 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cce5b6-b6a2-4f68-9e15-64d0a0e1919c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:22.340753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:22.414276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:22.432373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55991607-f23a-4388-bb0d-2df9366bf2e2 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:22.440754 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cce5b6-b6a2-4f68-9e15-64d0a0e1919c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:22.468080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff278b19-19fc-4523-8042-654b67ddedad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:22.468575 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff278b19-19fc-4523-8042-654b67ddedad None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:22.468854 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff278b19-19fc-4523-8042-654b67ddedad None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:22.469161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff278b19-19fc-4523-8042-654b67ddedad None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:22.469703 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff278b19-19fc-4523-8042-654b67ddedad None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:22.471107 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff278b19-19fc-4523-8042-654b67ddedad None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:22.480326 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e54af0-f935-473d-b6a6-6689d15c3df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:22.480635 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e54af0-f935-473d-b6a6-6689d15c3df0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:22.480734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ff278b19-19fc-4523-8042-654b67ddedad None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:14:22.480734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:22.480734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:22.480734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:22.480734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:22.480734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:22.480734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:22.480734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:22.480734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:14:22.480734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:22.480734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:22.480734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:22.480734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:22.480734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:22.480734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:22.480734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:22.481609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:22.482554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:22.482554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:22.482554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:22.482554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:22.482554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:22.482554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:22.482554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:22.482554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:22.482554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:22.482554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:22.482554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:22.482554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:22.482554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:14:22.482554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:22.483275 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e54af0-f935-473d-b6a6-6689d15c3df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:22.483275 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e54af0-f935-473d-b6a6-6689d15c3df0 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:22.483275 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e54af0-f935-473d-b6a6-6689d15c3df0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:22.483275 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e54af0-f935-473d-b6a6-6689d15c3df0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:22.483513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e54af0-f935-473d-b6a6-6689d15c3df0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:22.486711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 434/863] 10.222.0.76 () {56 vars in 1132 bytes} [Wed Aug 24 15:14:22 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:22.487567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff278b19-19fc-4523-8042-654b67ddedad None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:22.496058 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 430/864] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:22.501441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e54af0-f935-473d-b6a6-6689d15c3df0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:22.591309 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:22.604087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff278b19-19fc-4523-8042-654b67ddedad None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:22.610716 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e54af0-f935-473d-b6a6-6689d15c3df0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:22.641688 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efaeac-6c0e-482e-b53b-cc418cbfaa78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:22.642314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efaeac-6c0e-482e-b53b-cc418cbfaa78 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:22.642623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efaeac-6c0e-482e-b53b-cc418cbfaa78 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:22.642857 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06efaeac-6c0e-482e-b53b-cc418cbfaa78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:22.643579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efaeac-6c0e-482e-b53b-cc418cbfaa78 None admin] RBAC: Adding query filter params (role.id=59a2b50693a646fc8332b835233486ce, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=ad682eade4104a799b21f47aa7cdd4f5) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:22.643881 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efaeac-6c0e-482e-b53b-cc418cbfaa78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:22.645364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06efaeac-6c0e-482e-b53b-cc418cbfaa78 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:22.648108 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadc282-4004-4218-a4fa-9202778a826c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:22.648724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadc282-4004-4218-a4fa-9202778a826c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:22.648997 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadc282-4004-4218-a4fa-9202778a826c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:22.649270 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cadc282-4004-4218-a4fa-9202778a826c None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:22.649900 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadc282-4004-4218-a4fa-9202778a826c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:22.650200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadc282-4004-4218-a4fa-9202778a826c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:22.652001 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cadc282-4004-4218-a4fa-9202778a826c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:22.652513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 435/865] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:14:22 2022] GET /identity/v3/role_assignments?user.id=ad682eade4104a799b21f47aa7cdd4f5&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=59a2b50693a646fc8332b835233486ce => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:22.653325 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efaeac-6c0e-482e-b53b-cc418cbfaa78 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:22.661027 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 431/866] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 15:14:22 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:22.667628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadc282-4004-4218-a4fa-9202778a826c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:22.670792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:22.787648 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cadc282-4004-4218-a4fa-9202778a826c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:22.813115 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06efaeac-6c0e-482e-b53b-cc418cbfaa78 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:22.829540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5e448-3531-474c-a93a-5b482f72ac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:22.830166 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5e448-3531-474c-a93a-5b482f72ac54 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:22.830484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5e448-3531-474c-a93a-5b482f72ac54 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:22.830761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5e448-3531-474c-a93a-5b482f72ac54 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:22.831399 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5e448-3531-474c-a93a-5b482f72ac54 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:22.831706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5e448-3531-474c-a93a-5b482f72ac54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:22.833705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5e448-3531-474c-a93a-5b482f72ac54 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:22.839535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 432/867] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 15:14:22 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:22.858487 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84673e36-76a0-4a25-b263-7c89a68e8575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:22.859137 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84673e36-76a0-4a25-b263-7c89a68e8575 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:22.859416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84673e36-76a0-4a25-b263-7c89a68e8575 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:22.859685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84673e36-76a0-4a25-b263-7c89a68e8575 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:22.867713 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84673e36-76a0-4a25-b263-7c89a68e8575 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:22.869200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84673e36-76a0-4a25-b263-7c89a68e8575 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:22.888623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-84673e36-76a0-4a25-b263-7c89a68e8575 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:22.888623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:22.888623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:22.888623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:22.888623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:22.888623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:22.888623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:22.888623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:22.888623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:22.888623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:22.888623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:22.888623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:22.888623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:22.888623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:22.888623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:22.889624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:22.891143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:22.891143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:22.891143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:22.891143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:22.891143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:22.891143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:22.891143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:22.891143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:22.891143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 436/868] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:22.897051 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5e448-3531-474c-a93a-5b482f72ac54 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:22.985948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:23.040508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5e448-3531-474c-a93a-5b482f72ac54 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:23.077750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928c34f9-e76d-4a8f-bd37-ff7a14c5683f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:23.078825 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c34f9-e76d-4a8f-bd37-ff7a14c5683f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:23.079315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c34f9-e76d-4a8f-bd37-ff7a14c5683f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:23.079791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928c34f9-e76d-4a8f-bd37-ff7a14c5683f None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:23.080690 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c34f9-e76d-4a8f-bd37-ff7a14c5683f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:23.081222 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c34f9-e76d-4a8f-bd37-ff7a14c5683f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:23.083320 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928c34f9-e76d-4a8f-bd37-ff7a14c5683f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:23.104196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 433/869] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:23.105890 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84673e36-76a0-4a25-b263-7c89a68e8575 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:23.123301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:23.218757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84673e36-76a0-4a25-b263-7c89a68e8575 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:23.255952 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342c7cbf-2aaf-48b6-bc2d-bf7eda6a1f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:23.256567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c7cbf-2aaf-48b6-bc2d-bf7eda6a1f63 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:23.256840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c7cbf-2aaf-48b6-bc2d-bf7eda6a1f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:23.257103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342c7cbf-2aaf-48b6-bc2d-bf7eda6a1f63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:23.257847 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c7cbf-2aaf-48b6-bc2d-bf7eda6a1f63 None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=0679072b37db41e880a95b738bea47ea) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:23.258148 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c7cbf-2aaf-48b6-bc2d-bf7eda6a1f63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:23.259658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342c7cbf-2aaf-48b6-bc2d-bf7eda6a1f63 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:23.266841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 437/870] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:14:23 2022] GET /identity/v3/role_assignments?user.id=0679072b37db41e880a95b738bea47ea&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:24.107477 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a414d-2167-44bc-94ba-999939e85f18 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:24.108052 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a414d-2167-44bc-94ba-999939e85f18 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:24.108052 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15a414d-2167-44bc-94ba-999939e85f18 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:24.109591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 434/871] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:24.115292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bf5e4-ac41-47ee-a901-6b13a8d229ba None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:24.115703 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bf5e4-ac41-47ee-a901-6b13a8d229ba None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:24.115883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bf5e4-ac41-47ee-a901-6b13a8d229ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:24.134971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6f8bf5e4-ac41-47ee-a901-6b13a8d229ba None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:24.136274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6f8bf5e4-ac41-47ee-a901-6b13a8d229ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:24.210485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 438/872] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:24.487026 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a414d-2167-44bc-94ba-999939e85f18 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:24.610767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15a414d-2167-44bc-94ba-999939e85f18 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:24.638848 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:24.647445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc01fb2a-6c68-4ba3-bed5-7a84df63baec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:24.648059 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01fb2a-6c68-4ba3-bed5-7a84df63baec None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:24.648333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01fb2a-6c68-4ba3-bed5-7a84df63baec None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:24.648616 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc01fb2a-6c68-4ba3-bed5-7a84df63baec None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:24.649254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01fb2a-6c68-4ba3-bed5-7a84df63baec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:24.650646 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc01fb2a-6c68-4ba3-bed5-7a84df63baec None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:24.669152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 435/873] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 15:14:24 2022] POST /identity/v3/services => generated 251 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:14:24.796300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef52ebe-188b-40f5-9298-592de6ec4071 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:24.796720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef52ebe-188b-40f5-9298-592de6ec4071 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:24.797106 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef52ebe-188b-40f5-9298-592de6ec4071 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:24.798955 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 439/874] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:24.805038 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49699248-e35b-4465-bebd-05cf417d851c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:24.805502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49699248-e35b-4465-bebd-05cf417d851c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:24.805871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49699248-e35b-4465-bebd-05cf417d851c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:24.823816 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4f064-2bf5-4203-80ec-35e3891f5a6e None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:24.824093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4f064-2bf5-4203-80ec-35e3891f5a6e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:24.824364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4f064-2bf5-4203-80ec-35e3891f5a6e None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:24.825457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-49699248-e35b-4465-bebd-05cf417d851c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:24.826054 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 440/875] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:14:24.826792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-49699248-e35b-4465-bebd-05cf417d851c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:24.831489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b05d7-9d04-44a2-ad40-c21027753706 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:24.831777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b05d7-9d04-44a2-ad40-c21027753706 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:24.832030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641b05d7-9d04-44a2-ad40-c21027753706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:24.850040 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:24.859442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-641b05d7-9d04-44a2-ad40-c21027753706 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:24.860644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-641b05d7-9d04-44a2-ad40-c21027753706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:24.929840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 436/876] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:24 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:24.955527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 441/877] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:24 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:25.199260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49699248-e35b-4465-bebd-05cf417d851c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:25.237366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b05d7-9d04-44a2-ad40-c21027753706 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:25.313859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49699248-e35b-4465-bebd-05cf417d851c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:25.331468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:25.355342 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca432f1-f71f-4ba7-b871-37c167b4609b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:25.355977 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca432f1-f71f-4ba7-b871-37c167b4609b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:25.356253 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca432f1-f71f-4ba7-b871-37c167b4609b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:25.356520 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca432f1-f71f-4ba7-b871-37c167b4609b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:25.365055 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641b05d7-9d04-44a2-ad40-c21027753706 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:25.366893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca432f1-f71f-4ba7-b871-37c167b4609b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:25.371615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7ca432f1-f71f-4ba7-b871-37c167b4609b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:14:25.371615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:25.371615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:25.371615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:25.371615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:25.371615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:25.371615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:25.371615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:25.371615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:25.371615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:25.371615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:25.371615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:25.371615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:25.371615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:25.371615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:25.371615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:25.372586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:25.372586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:25.372586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:25.372586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:25.372586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:25.372586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:25.372586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:25.372586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:25.372586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:25.372586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:25.372586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:25.372586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:25.372586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:25.372586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:25.372586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:25.372586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:25.375203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:25.375203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:25.375203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:25.375203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:25.375203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:25.375203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:25.375203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:25.375203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:25.375203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:25.375203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:25.375203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:14:25.375203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:25.375203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 437/878] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 24 15:14:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:25.375203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54346d58-dcc9-42dc-a039-fc0753ba3770 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:25.380785 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54346d58-dcc9-42dc-a039-fc0753ba3770 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:25.380785 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54346d58-dcc9-42dc-a039-fc0753ba3770 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:25.380785 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 438/879] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:25.380785 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54346d58-dcc9-42dc-a039-fc0753ba3770 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:25.413532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f17d85-aacc-462d-bd23-bd625adfed95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:25.414142 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f17d85-aacc-462d-bd23-bd625adfed95 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:25.414455 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f17d85-aacc-462d-bd23-bd625adfed95 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:25.414733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f17d85-aacc-462d-bd23-bd625adfed95 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:25.415378 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f17d85-aacc-462d-bd23-bd625adfed95 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:25.416785 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f17d85-aacc-462d-bd23-bd625adfed95 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:25.421029 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:25.437456 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 442/880] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 15:14:25 2022] POST /identity/v3/services => generated 269 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:14:25.438965 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e8ad7-b27b-4a88-bd65-40e36055105b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:25.439405 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e8ad7-b27b-4a88-bd65-40e36055105b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:25.439811 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813e8ad7-b27b-4a88-bd65-40e36055105b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:25.461497 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-813e8ad7-b27b-4a88-bd65-40e36055105b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:25.463137 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-813e8ad7-b27b-4a88-bd65-40e36055105b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:25.522763 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54346d58-dcc9-42dc-a039-fc0753ba3770 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:25.564387 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083eeb71-9ab1-4c31-8673-4b5da2c788e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:25.565030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083eeb71-9ab1-4c31-8673-4b5da2c788e3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:25.565256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083eeb71-9ab1-4c31-8673-4b5da2c788e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:25.565502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 443/881] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:25 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:25.565606 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083eeb71-9ab1-4c31-8673-4b5da2c788e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:25.569857 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083eeb71-9ab1-4c31-8673-4b5da2c788e3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:25.569857 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083eeb71-9ab1-4c31-8673-4b5da2c788e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:25.569857 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083eeb71-9ab1-4c31-8673-4b5da2c788e3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:25.579100 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 439/882] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 24 15:14:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:25.585250 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e8ad7-b27b-4a88-bd65-40e36055105b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:25.695336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813e8ad7-b27b-4a88-bd65-40e36055105b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:25.723179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:25.733303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721beef9-88a0-4522-88cd-aef93c4c535c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:25.733945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721beef9-88a0-4522-88cd-aef93c4c535c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:25.734236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721beef9-88a0-4522-88cd-aef93c4c535c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:25.734508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721beef9-88a0-4522-88cd-aef93c4c535c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:25.746036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721beef9-88a0-4522-88cd-aef93c4c535c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:25.747527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721beef9-88a0-4522-88cd-aef93c4c535c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:25.763206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-721beef9-88a0-4522-88cd-aef93c4c535c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:25.763206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:25.763206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:25.763206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:25.763206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:25.763206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:25.763206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:25.763206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:25.763206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:25.763206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:25.763206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:25.763206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:25.763206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:25.763206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:25.763206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:25.763206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:25.764164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:25.765047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:25.765047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:25.765047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:25.765047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 444/883] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:25.775383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083eeb71-9ab1-4c31-8673-4b5da2c788e3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:25.810190 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:25.853052 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721beef9-88a0-4522-88cd-aef93c4c535c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:25.896781 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083eeb71-9ab1-4c31-8673-4b5da2c788e3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:25.938990 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45053c-d749-443b-a638-32e5f20800ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:25.939601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45053c-d749-443b-a638-32e5f20800ef None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:25.939878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45053c-d749-443b-a638-32e5f20800ef None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:25.940145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c45053c-d749-443b-a638-32e5f20800ef None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:25.940790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45053c-d749-443b-a638-32e5f20800ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:25.941202 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45053c-d749-443b-a638-32e5f20800ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:25.942646 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c45053c-d749-443b-a638-32e5f20800ef None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:25.956156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 440/884] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:25.961952 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45053c-d749-443b-a638-32e5f20800ef None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:25.972186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721beef9-88a0-4522-88cd-aef93c4c535c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:26.018526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7463a0-b51a-409e-8838-4170dee210fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:26.019027 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7463a0-b51a-409e-8838-4170dee210fc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:26.019293 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7463a0-b51a-409e-8838-4170dee210fc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:26.021619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7463a0-b51a-409e-8838-4170dee210fc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:26.021619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7463a0-b51a-409e-8838-4170dee210fc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:26.024617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7463a0-b51a-409e-8838-4170dee210fc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-fe7463a0-b51a-409e-8838-4170dee210fc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:26.037267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:26.038443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:26.038443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:26.038443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:26.038443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:26.038443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:26.038443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:26.038443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:26.038443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:26.038443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:26.038443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:26.038443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:26.038443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:26.038443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:26.038443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:26.038443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:26.038443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:26.039340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:26.039340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:26.039340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:26.039340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:26.039340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:26.039340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:26.039340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:26.039340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:26.039340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:26.039340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:26.039340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:26.039340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:26.039340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:14:26.039340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:26.039340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 445/885] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 15:14:25 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:26.044528 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7463a0-b51a-409e-8838-4170dee210fc None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:26.102547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:26.109825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:26.119715 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c45053c-d749-443b-a638-32e5f20800ef None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:26.169639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b510656-fbbf-4b71-afb6-71699e9f0d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:26.170989 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b510656-fbbf-4b71-afb6-71699e9f0d7a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:26.171370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b510656-fbbf-4b71-afb6-71699e9f0d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:26.171827 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b510656-fbbf-4b71-afb6-71699e9f0d7a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:26.184104 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b510656-fbbf-4b71-afb6-71699e9f0d7a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:26.185420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7463a0-b51a-409e-8838-4170dee210fc None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:26.185944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b510656-fbbf-4b71-afb6-71699e9f0d7a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:26.203236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6b510656-fbbf-4b71-afb6-71699e9f0d7a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:14:26.203236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:26.203236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:26.203236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:26.203236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:26.203236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:26.203236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:26.203236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:26.203236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:26.203236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:26.203236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:26.203236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:26.203236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:26.203236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:26.203236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:26.203236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:26.204909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:26.205925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:26.205925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:26.205925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:26.205925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:26.205925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:26.205925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:26.205925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:26.205925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:26.205925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:14:26.205925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:26.214457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 441/886] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 15:14:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:26.214844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b510656-fbbf-4b71-afb6-71699e9f0d7a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:26.237187 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea838c7-0ef2-4286-b897-406c1af873cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:26.237810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea838c7-0ef2-4286-b897-406c1af873cb None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:26.238417 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea838c7-0ef2-4286-b897-406c1af873cb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:26.238417 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea838c7-0ef2-4286-b897-406c1af873cb None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:26.239028 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea838c7-0ef2-4286-b897-406c1af873cb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:26.239330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea838c7-0ef2-4286-b897-406c1af873cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:26.240699 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea838c7-0ef2-4286-b897-406c1af873cb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:26.249518 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 446/887] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 15:14:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:26.255487 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea838c7-0ef2-4286-b897-406c1af873cb None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:26.358855 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b510656-fbbf-4b71-afb6-71699e9f0d7a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:26.372078 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea838c7-0ef2-4286-b897-406c1af873cb None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:26.379648 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:26.403064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55643311-981c-4372-bcd6-4ec9f4890eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:26.404187 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55643311-981c-4372-bcd6-4ec9f4890eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:26.404693 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55643311-981c-4372-bcd6-4ec9f4890eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:26.405200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55643311-981c-4372-bcd6-4ec9f4890eb5 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:26.406336 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55643311-981c-4372-bcd6-4ec9f4890eb5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:26.406948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55643311-981c-4372-bcd6-4ec9f4890eb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:26.408728 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55643311-981c-4372-bcd6-4ec9f4890eb5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:26.412292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f411e-4711-45dc-8491-0138e918e2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:26.413601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f411e-4711-45dc-8491-0138e918e2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:26.413601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f411e-4711-45dc-8491-0138e918e2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:26.413601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f411e-4711-45dc-8491-0138e918e2c7 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:26.414039 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f411e-4711-45dc-8491-0138e918e2c7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:26.414362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f411e-4711-45dc-8491-0138e918e2c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:26.415743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f411e-4711-45dc-8491-0138e918e2c7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:26.430256 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 447/888] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 15:14:26 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:26.443139 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 442/889] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 24 15:14:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:26.448581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f411e-4711-45dc-8491-0138e918e2c7 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:26.591209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f411e-4711-45dc-8491-0138e918e2c7 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:26.631129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6231039-e12f-4b66-97fa-8a0b251a1b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:26.631669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6231039-e12f-4b66-97fa-8a0b251a1b73 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:26.631950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6231039-e12f-4b66-97fa-8a0b251a1b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:26.632226 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6231039-e12f-4b66-97fa-8a0b251a1b73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:26.674989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:26.709639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6231039-e12f-4b66-97fa-8a0b251a1b73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:26.711127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6231039-e12f-4b66-97fa-8a0b251a1b73 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:26.723845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967aee38-dc20-4cf9-b1f3-b68e2e8d262e None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:26.724164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967aee38-dc20-4cf9-b1f3-b68e2e8d262e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:26.724387 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967aee38-dc20-4cf9-b1f3-b68e2e8d262e None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:26.725927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 443/890] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:26.732013 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616897f4-6652-480e-bf92-303e7b18329b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:26.732238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616897f4-6652-480e-bf92-303e7b18329b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:26.732516 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616897f4-6652-480e-bf92-303e7b18329b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:26.757714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-616897f4-6652-480e-bf92-303e7b18329b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:26.759034 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-616897f4-6652-480e-bf92-303e7b18329b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:26.762774 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:26.834339 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 448/891] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:14:26 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:14:26.842298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6231039-e12f-4b66-97fa-8a0b251a1b73 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:26.852710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 444/892] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:26 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:26.921631 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:26.962708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6231039-e12f-4b66-97fa-8a0b251a1b73 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:26.999731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4d440-48f6-4503-9ca7-d26f61f140cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:27.000362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4d440-48f6-4503-9ca7-d26f61f140cc None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:27.000639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4d440-48f6-4503-9ca7-d26f61f140cc None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:27.000914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4d440-48f6-4503-9ca7-d26f61f140cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:27.012719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4d440-48f6-4503-9ca7-d26f61f140cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:27.014179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4d440-48f6-4503-9ca7-d26f61f140cc None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:27.029825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-bfc4d440-48f6-4503-9ca7-d26f61f140cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:27.029825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:27.029825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:27.029825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:27.029825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:27.029825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:27.029825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:27.029825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:27.029825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:27.029825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:27.029825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:27.029825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:27.029825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:27.029825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:27.029825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:27.029825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:27.030861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:27.031728 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:27.031728 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:27.031728 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:27.031728 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 449/893] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:27.036959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616897f4-6652-480e-bf92-303e7b18329b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:27.143244 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4d440-48f6-4503-9ca7-d26f61f140cc None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:27.167433 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616897f4-6652-480e-bf92-303e7b18329b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:27.206835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee329266-284c-4c8b-85d1-6e3a90d81b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:27.208302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee329266-284c-4c8b-85d1-6e3a90d81b84 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:27.208799 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee329266-284c-4c8b-85d1-6e3a90d81b84 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:27.209262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee329266-284c-4c8b-85d1-6e3a90d81b84 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:27.210262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee329266-284c-4c8b-85d1-6e3a90d81b84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:27.210920 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee329266-284c-4c8b-85d1-6e3a90d81b84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:27.212531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee329266-284c-4c8b-85d1-6e3a90d81b84 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:27.220268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:27.228028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 445/894] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:27.236375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee329266-284c-4c8b-85d1-6e3a90d81b84 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:27.266515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4d440-48f6-4503-9ca7-d26f61f140cc None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:27.298896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:27.318252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0c119-ef84-4692-af16-b19b416e1ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:27.319121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0c119-ef84-4692-af16-b19b416e1ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:27.319406 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0c119-ef84-4692-af16-b19b416e1ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:27.319677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a0c119-ef84-4692-af16-b19b416e1ad5 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:27.320270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0c119-ef84-4692-af16-b19b416e1ad5 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:27.321740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a0c119-ef84-4692-af16-b19b416e1ad5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-66a0c119-ef84-4692-af16-b19b416e1ad5 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:27.333753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:27.334863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:27.334863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:27.334863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:27.334863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:27.334863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:27.334863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:27.334863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:27.334863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:27.334863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:27.334863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:27.334863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:27.334863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:27.334863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:27.334863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:27.334863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:27.334863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:27.335680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:27.335680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:27.335680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:27.335680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:27.335680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:27.335680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:27.335680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:27.335680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:27.335680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:27.335680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:27.335680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:27.335680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:27.335680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:14:27.335680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:27.335680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 450/895] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:14:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:27.340946 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0c119-ef84-4692-af16-b19b416e1ad5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:27.356993 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee329266-284c-4c8b-85d1-6e3a90d81b84 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:27.393889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248ab30-1063-428c-8ba8-a9e48722f733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:27.395150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248ab30-1063-428c-8ba8-a9e48722f733 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:27.395736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248ab30-1063-428c-8ba8-a9e48722f733 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:27.396347 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c248ab30-1063-428c-8ba8-a9e48722f733 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:27.404580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248ab30-1063-428c-8ba8-a9e48722f733 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:27.406452 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c248ab30-1063-428c-8ba8-a9e48722f733 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:27.419806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c248ab30-1063-428c-8ba8-a9e48722f733 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:27.419806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:27.419806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:27.419806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:27.419806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:27.419806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:27.419806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:27.419806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:27.419806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:27.419806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:27.419806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:27.419806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:27.419806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:27.419806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:27.419806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:27.420815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:27.421698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:27.421698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:27.421698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:27.421698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:27.421698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:27.421698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:27.421698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:27.421698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:27.424114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 446/896] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:27.430688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248ab30-1063-428c-8ba8-a9e48722f733 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:27.490951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a0c119-ef84-4692-af16-b19b416e1ad5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:27.498488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:27.549298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265afca-3861-44f2-b4e8-cb3e0c83b7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:27.550143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265afca-3861-44f2-b4e8-cb3e0c83b7d0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:27.550456 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265afca-3861-44f2-b4e8-cb3e0c83b7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:27.550720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d265afca-3861-44f2-b4e8-cb3e0c83b7d0 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:27.551907 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265afca-3861-44f2-b4e8-cb3e0c83b7d0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:27.552227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265afca-3861-44f2-b4e8-cb3e0c83b7d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:27.554336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d265afca-3861-44f2-b4e8-cb3e0c83b7d0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:27.555822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c248ab30-1063-428c-8ba8-a9e48722f733 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:27.560809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 451/897] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:14:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:27.565920 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265afca-3861-44f2-b4e8-cb3e0c83b7d0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:27.599760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39485d0-817c-45b8-be22-f3cc3009afb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:27.600430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39485d0-817c-45b8-be22-f3cc3009afb3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:27.600717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39485d0-817c-45b8-be22-f3cc3009afb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:27.600988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39485d0-817c-45b8-be22-f3cc3009afb3 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:27.601681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39485d0-817c-45b8-be22-f3cc3009afb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:27.601985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39485d0-817c-45b8-be22-f3cc3009afb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:27.603449 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39485d0-817c-45b8-be22-f3cc3009afb3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:27.621362 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 447/898] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:27.623864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f30695-ea4c-4bc6-bfd6-5b1bb60c0b05 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:27.624355 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f30695-ea4c-4bc6-bfd6-5b1bb60c0b05 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:27.624809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f30695-ea4c-4bc6-bfd6-5b1bb60c0b05 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:27.626527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 448/899] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:27.627943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f30695-ea4c-4bc6-bfd6-5b1bb60c0b05 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:27.631190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:27.688487 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d265afca-3861-44f2-b4e8-cb3e0c83b7d0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:27.726448 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ab1d54-98c2-42a8-b33d-cc2481bd5d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:27.727051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab1d54-98c2-42a8-b33d-cc2481bd5d77 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:27.727327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab1d54-98c2-42a8-b33d-cc2481bd5d77 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:27.727589 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ab1d54-98c2-42a8-b33d-cc2481bd5d77 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:27.728200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab1d54-98c2-42a8-b33d-cc2481bd5d77 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:27.728500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab1d54-98c2-42a8-b33d-cc2481bd5d77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:27.729849 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ab1d54-98c2-42a8-b33d-cc2481bd5d77 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:27.734935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 452/900] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:14:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:27.739777 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f30695-ea4c-4bc6-bfd6-5b1bb60c0b05 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:27.741144 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ef186-6847-4cf8-985e-a9773b04a3e0 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:27.741377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ef186-6847-4cf8-985e-a9773b04a3e0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:27.741647 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ef186-6847-4cf8-985e-a9773b04a3e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:27.759363 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:27.759797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4b2ef186-6847-4cf8-985e-a9773b04a3e0 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:27.760969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4b2ef186-6847-4cf8-985e-a9773b04a3e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:27.778015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1626563-ecb0-4d77-8399-4c432248896e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:27.778646 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1626563-ecb0-4d77-8399-4c432248896e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:27.778923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1626563-ecb0-4d77-8399-4c432248896e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:27.779194 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1626563-ecb0-4d77-8399-4c432248896e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:27.779916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1626563-ecb0-4d77-8399-4c432248896e None admin] RBAC: Adding query filter params (role.id=c87fe7f37cb246e8af3bc71057ec5048, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=ad682eade4104a799b21f47aa7cdd4f5) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:27.780221 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1626563-ecb0-4d77-8399-4c432248896e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:27.781634 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1626563-ecb0-4d77-8399-4c432248896e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:27.794255 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 449/901] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:14:27 2022] GET /identity/v3/role_assignments?user.id=ad682eade4104a799b21f47aa7cdd4f5&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=c87fe7f37cb246e8af3bc71057ec5048 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:27.794255 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1626563-ecb0-4d77-8399-4c432248896e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:27.847649 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 453/902] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:27.906848 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1626563-ecb0-4d77-8399-4c432248896e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:27.948423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315566b-57d8-41ab-9406-c9b76d3e7046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:27.949121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315566b-57d8-41ab-9406-c9b76d3e7046 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:27.949405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315566b-57d8-41ab-9406-c9b76d3e7046 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:27.949677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6315566b-57d8-41ab-9406-c9b76d3e7046 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:27.950611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315566b-57d8-41ab-9406-c9b76d3e7046 None admin] RBAC: Adding query filter params (interface=public, service_id=f9195918a4c74dd9bf33c8b3866d275a, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:27.951032 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315566b-57d8-41ab-9406-c9b76d3e7046 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:27.952450 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6315566b-57d8-41ab-9406-c9b76d3e7046 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:27.960679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 450/903] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 24 15:14:27 2022] GET /identity/v3/endpoints?service_id=f9195918a4c74dd9bf33c8b3866d275a&interface=public®ion_id=RegionOne => generated 194 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:27.966266 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ef186-6847-4cf8-985e-a9773b04a3e0 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:28.023083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:28.088323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ef186-6847-4cf8-985e-a9773b04a3e0 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:28.124768 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0d005-45a5-4b10-ab5e-3e26e1183e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:28.125793 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0d005-45a5-4b10-ab5e-3e26e1183e0c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:28.126303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0d005-45a5-4b10-ab5e-3e26e1183e0c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:28.126793 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0d005-45a5-4b10-ab5e-3e26e1183e0c None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:28.127606 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0d005-45a5-4b10-ab5e-3e26e1183e0c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:28.129818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be0d005-45a5-4b10-ab5e-3e26e1183e0c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:28.136341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 454/904] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 15:14:27 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:14:28.175528 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315566b-57d8-41ab-9406-c9b76d3e7046 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:28.230015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:28.292830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6315566b-57d8-41ab-9406-c9b76d3e7046 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:28.339769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81302df4-2d3b-4a45-97ca-f37826208cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:28.340422 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81302df4-2d3b-4a45-97ca-f37826208cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:28.340706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81302df4-2d3b-4a45-97ca-f37826208cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:28.340984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81302df4-2d3b-4a45-97ca-f37826208cf8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:28.341575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81302df4-2d3b-4a45-97ca-f37826208cf8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:28.342978 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81302df4-2d3b-4a45-97ca-f37826208cf8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:28.352940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-81302df4-2d3b-4a45-97ca-f37826208cf8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:14:28.352940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:28.352940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:28.352940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:28.352940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:28.352940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:28.352940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:28.352940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:28.352940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:14:28.352940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:28.352940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:28.352940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:28.352940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:28.352940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:28.352940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:28.352940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:28.353930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:28.354876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:28.354876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:28.354876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:28.354876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:28.354876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:28.354876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:28.354876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:28.354876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:28.354876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:28.354876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:28.354876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:28.354876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:28.354876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:14:28.354876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:28.354876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 451/905] 10.222.0.76 () {56 vars in 1132 bytes} [Wed Aug 24 15:14:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:28.367548 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0d005-45a5-4b10-ab5e-3e26e1183e0c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:28.478492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0d005-45a5-4b10-ab5e-3e26e1183e0c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:28.489786 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:28.518057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411aefbc-8972-42b5-ac72-48f1c852397c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:28.518763 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411aefbc-8972-42b5-ac72-48f1c852397c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:28.519046 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411aefbc-8972-42b5-ac72-48f1c852397c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:28.519318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411aefbc-8972-42b5-ac72-48f1c852397c None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:28.520540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411aefbc-8972-42b5-ac72-48f1c852397c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:28.520844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411aefbc-8972-42b5-ac72-48f1c852397c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:28.522236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411aefbc-8972-42b5-ac72-48f1c852397c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:28.528431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 455/906] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 15:14:28 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:28.534423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81302df4-2d3b-4a45-97ca-f37826208cf8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:28.573184 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb16a4e-3392-427d-a623-3070055bcbcb None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:28.573384 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb16a4e-3392-427d-a623-3070055bcbcb None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:28.573635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb16a4e-3392-427d-a623-3070055bcbcb None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:28.575373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 456/907] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:14:28.581344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362da2eb-d822-43c5-8765-c850011c4c8b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:28.581627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362da2eb-d822-43c5-8765-c850011c4c8b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:28.581900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362da2eb-d822-43c5-8765-c850011c4c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:28.601576 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-362da2eb-d822-43c5-8765-c850011c4c8b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:28.602817 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-362da2eb-d822-43c5-8765-c850011c4c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:28.661229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81302df4-2d3b-4a45-97ca-f37826208cf8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:28.673331 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:28.706932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 457/908] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:28 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:28.708774 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13c3fb-1fb1-4a7f-b707-9a856916af88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:28.709390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13c3fb-1fb1-4a7f-b707-9a856916af88 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:28.709669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13c3fb-1fb1-4a7f-b707-9a856916af88 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:28.709942 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13c3fb-1fb1-4a7f-b707-9a856916af88 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:28.710624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13c3fb-1fb1-4a7f-b707-9a856916af88 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:28.710926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13c3fb-1fb1-4a7f-b707-9a856916af88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:28.712348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13c3fb-1fb1-4a7f-b707-9a856916af88 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:28.718101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 452/909] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 15:14:28 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:28.728468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362da2eb-d822-43c5-8765-c850011c4c8b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:28.839125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362da2eb-d822-43c5-8765-c850011c4c8b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:28.840564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:28.877436 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e0835-f045-48ff-b6f7-d4a5d6d24f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:28.878053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e0835-f045-48ff-b6f7-d4a5d6d24f06 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:28.878358 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e0835-f045-48ff-b6f7-d4a5d6d24f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:28.878625 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1e0835-f045-48ff-b6f7-d4a5d6d24f06 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:28.879893 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e0835-f045-48ff-b6f7-d4a5d6d24f06 None admin] RBAC: Adding query filter params (interface=public, service_id=5a74a668a92f4db693d717fa6fe22bd5, region_id=RegionOne) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:28.880193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e0835-f045-48ff-b6f7-d4a5d6d24f06 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:28.881605 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1e0835-f045-48ff-b6f7-d4a5d6d24f06 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:28.889563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 458/910] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 24 15:14:28 2022] GET /identity/v3/endpoints?service_id=5a74a668a92f4db693d717fa6fe22bd5&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:28.895867 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13c3fb-1fb1-4a7f-b707-9a856916af88 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:28.990570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e0835-f045-48ff-b6f7-d4a5d6d24f06 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:29.010262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13c3fb-1fb1-4a7f-b707-9a856916af88 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:29.047250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108bcd5f-4944-4cb8-a092-13f4e9be1d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:29.047864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bcd5f-4944-4cb8-a092-13f4e9be1d89 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:29.048139 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bcd5f-4944-4cb8-a092-13f4e9be1d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:29.048406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108bcd5f-4944-4cb8-a092-13f4e9be1d89 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:29.049010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108bcd5f-4944-4cb8-a092-13f4e9be1d89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:29.050380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108bcd5f-4944-4cb8-a092-13f4e9be1d89 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:29.057266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 453/911] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 15:14:28 2022] GET /identity/v3/services => generated 1268 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:14:29.124759 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1e0835-f045-48ff-b6f7-d4a5d6d24f06 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:29.166306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7714b20b-1090-4ff5-8423-934fe711d1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:29.166931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714b20b-1090-4ff5-8423-934fe711d1e1 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:29.167205 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714b20b-1090-4ff5-8423-934fe711d1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:29.167471 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7714b20b-1090-4ff5-8423-934fe711d1e1 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:29.168135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714b20b-1090-4ff5-8423-934fe711d1e1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:29.169520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7714b20b-1090-4ff5-8423-934fe711d1e1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:29.186974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 459/912] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 15:14:28 2022] POST /identity/v3/services => generated 241 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:14:30.233361 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387c3fd-7a42-449f-bfaa-151f377a7496 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:30.233892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387c3fd-7a42-449f-bfaa-151f377a7496 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:30.234315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0387c3fd-7a42-449f-bfaa-151f377a7496 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:30.236828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 454/913] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:30.243139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774defa-78a7-4a20-b3b6-489a59a836aa None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:30.244038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774defa-78a7-4a20-b3b6-489a59a836aa None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:30.244542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3774defa-78a7-4a20-b3b6-489a59a836aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:30.260078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edffca7-27fe-4bfc-9b42-6b4d509d8e92 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:30.260379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edffca7-27fe-4bfc-9b42-6b4d509d8e92 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:30.260605 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edffca7-27fe-4bfc-9b42-6b4d509d8e92 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:30.262041 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 455/914] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:30.268537 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3774defa-78a7-4a20-b3b6-489a59a836aa None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:30.268763 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe08d46-d8a8-4f7b-bb58-06906027073d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:30.268763 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe08d46-d8a8-4f7b-bb58-06906027073d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:30.268763 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe08d46-d8a8-4f7b-bb58-06906027073d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:30.269220 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3774defa-78a7-4a20-b3b6-489a59a836aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:30.289547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4fe08d46-d8a8-4f7b-bb58-06906027073d None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:30.291112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4fe08d46-d8a8-4f7b-bb58-06906027073d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:30.292756 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:30.334622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:30.370361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 460/915] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:30.380399 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 456/916] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:30 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) Aug 24 15:14:30.645651 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774defa-78a7-4a20-b3b6-489a59a836aa None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:30.667075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe08d46-d8a8-4f7b-bb58-06906027073d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:30.778067 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3774defa-78a7-4a20-b3b6-489a59a836aa None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:30.797031 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe08d46-d8a8-4f7b-bb58-06906027073d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:30.821679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466a3b3-b37e-44db-8750-0719d9f40446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:30.822749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466a3b3-b37e-44db-8750-0719d9f40446 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:30.823172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466a3b3-b37e-44db-8750-0719d9f40446 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:30.823694 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7466a3b3-b37e-44db-8750-0719d9f40446 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:30.839744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7466a3b3-b37e-44db-8750-0719d9f40446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:30.843312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278dcf0-a1b7-4c47-a188-2c468b7e7373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:30.844069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278dcf0-a1b7-4c47-a188-2c468b7e7373 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:30.844437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278dcf0-a1b7-4c47-a188-2c468b7e7373 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:30.844798 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278dcf0-a1b7-4c47-a188-2c468b7e7373 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:30.845521 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278dcf0-a1b7-4c47-a188-2c468b7e7373 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:30.845914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-7466a3b3-b37e-44db-8750-0719d9f40446 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:14:30.845914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:30.845914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:30.845914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:30.845914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:30.845914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:30.845914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:30.845914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:30.845914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:30.845914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:30.845914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:30.845914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:30.845914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:30.845914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:30.845914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:30.845914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:30.846889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:30.846889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:30.846889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:30.846889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:30.846889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:30.846889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:30.846889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:30.846889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:30.846889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:30.846889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:30.846889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:30.846889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:30.846889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:30.846889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:30.846889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:30.846889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:30.848301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:30.848301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:30.848301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:30.848301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:30.848301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:30.848301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:30.848301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:30.848301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:30.848301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:30.848301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:30.848301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:14:30.848301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:30.849004 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278dcf0-a1b7-4c47-a188-2c468b7e7373 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:30.851217 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 461/917] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 24 15:14:30 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:30.859984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466a3b3-b37e-44db-8750-0719d9f40446 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0278dcf0-a1b7-4c47-a188-2c468b7e7373 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:30.866208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:30.868184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:30.868184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:30.868184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:30.868184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:30.868184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:30.868184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:30.868184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:30.868184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:30.868184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:30.868184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:30.868184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:30.868184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:30.868184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:30.868184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:30.868184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:30.868184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:30.869761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:30.869761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:30.869761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:30.869761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:30.869761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:30.869761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:30.869761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:30.869761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:30.869761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:30.869761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:30.869761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:30.869761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:30.869761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 15:14:30.869761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:30.875710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 457/918] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:14:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:30.878782 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278dcf0-a1b7-4c47-a188-2c468b7e7373 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:30.885085 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:30.926584 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:31.012880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7466a3b3-b37e-44db-8750-0719d9f40446 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:31.014986 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278dcf0-a1b7-4c47-a188-2c468b7e7373 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:31.056191 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c903963-2123-40b1-812e-0e74385a258c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:31.056730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c903963-2123-40b1-812e-0e74385a258c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:31.057005 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c903963-2123-40b1-812e-0e74385a258c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:31.057370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c903963-2123-40b1-812e-0e74385a258c None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:31.058144 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c903963-2123-40b1-812e-0e74385a258c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:31.058470 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c903963-2123-40b1-812e-0e74385a258c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:31.059742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920b002-ee89-4222-b1e4-8cd40611f77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:31.059947 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c903963-2123-40b1-812e-0e74385a258c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:31.060741 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920b002-ee89-4222-b1e4-8cd40611f77a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:31.061232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920b002-ee89-4222-b1e4-8cd40611f77a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:31.061697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920b002-ee89-4222-b1e4-8cd40611f77a None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:31.062551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920b002-ee89-4222-b1e4-8cd40611f77a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:31.063055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920b002-ee89-4222-b1e4-8cd40611f77a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:31.064638 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920b002-ee89-4222-b1e4-8cd40611f77a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:31.075147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 462/920] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 24 15:14:30 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:31.075147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c903963-2123-40b1-812e-0e74385a258c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:31.075602 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 458/920] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:14:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:31.076845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920b002-ee89-4222-b1e4-8cd40611f77a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:31.161166 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:31.180490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:31.190671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920b002-ee89-4222-b1e4-8cd40611f77a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:31.192761 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c903963-2123-40b1-812e-0e74385a258c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:31.231881 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e733e77-7b73-4f4b-96de-8b8495064143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:31.232112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3143bf78-d49d-4cb2-a832-c1c6d8396c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:31.232489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e733e77-7b73-4f4b-96de-8b8495064143 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:31.232619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143bf78-d49d-4cb2-a832-c1c6d8396c3b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:31.232787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e733e77-7b73-4f4b-96de-8b8495064143 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:31.232890 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143bf78-d49d-4cb2-a832-c1c6d8396c3b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:31.233065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e733e77-7b73-4f4b-96de-8b8495064143 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:31.233159 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3143bf78-d49d-4cb2-a832-c1c6d8396c3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:31.233643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e733e77-7b73-4f4b-96de-8b8495064143 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:31.234071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e733e77-7b73-4f4b-96de-8b8495064143 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:31.235308 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e733e77-7b73-4f4b-96de-8b8495064143 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:31.245575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 459/921] 10.222.0.76 () {56 vars in 1114 bytes} [Wed Aug 24 15:14:31 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:31.245575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386515a8-bfaa-4c1e-9061-712cc6e2520a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:31.245575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386515a8-bfaa-4c1e-9061-712cc6e2520a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:31.245575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386515a8-bfaa-4c1e-9061-712cc6e2520a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:31.245575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 460/922] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:31.247921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386515a8-bfaa-4c1e-9061-712cc6e2520a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:31.249305 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143bf78-d49d-4cb2-a832-c1c6d8396c3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:31.250983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3143bf78-d49d-4cb2-a832-c1c6d8396c3b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:31.265769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-3143bf78-d49d-4cb2-a832-c1c6d8396c3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:31.265769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:31.265769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:31.265769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:31.265769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:31.265769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:31.265769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:31.265769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:31.265769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:31.265769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:31.265769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:31.265769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:31.265769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:31.265769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:31.265769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:31.265769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:31.266934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:31.268111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:31.268111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:31.268111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:31.271996 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 463/923] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:31.273250 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d42c68-89f5-4b9e-9c40-b347d989efe2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:31.273535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d42c68-89f5-4b9e-9c40-b347d989efe2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:31.273804 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d42c68-89f5-4b9e-9c40-b347d989efe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:31.293284 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-58d42c68-89f5-4b9e-9c40-b347d989efe2 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:31.294551 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-58d42c68-89f5-4b9e-9c40-b347d989efe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:31.362992 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386515a8-bfaa-4c1e-9061-712cc6e2520a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:31.378242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 464/924] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:31.378242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d42c68-89f5-4b9e-9c40-b347d989efe2 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:31.401369 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c2e88-2e11-4107-9db9-889110fde11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:31.402147 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c2e88-2e11-4107-9db9-889110fde11c None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:31.402465 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c2e88-2e11-4107-9db9-889110fde11c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:31.402793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c2e88-2e11-4107-9db9-889110fde11c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:31.403833 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c2e88-2e11-4107-9db9-889110fde11c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:31.405501 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3c2e88-2e11-4107-9db9-889110fde11c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:31.433848 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 461/925] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:14:31 2022] POST /identity/v3/endpoints => generated 346 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:14:31.467150 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:31.474085 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90196b6-1e50-4088-a768-57b3c710ea58 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:31.474470 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90196b6-1e50-4088-a768-57b3c710ea58 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:31.474752 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90196b6-1e50-4088-a768-57b3c710ea58 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:31.477512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 462/926] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:14:31.484739 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a361a31-c5bb-4c57-a807-6e31671950ea None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:31.485958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a361a31-c5bb-4c57-a807-6e31671950ea None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:31.486445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a361a31-c5bb-4c57-a807-6e31671950ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:31.512757 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1a361a31-c5bb-4c57-a807-6e31671950ea None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:31.514114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1a361a31-c5bb-4c57-a807-6e31671950ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:31.526816 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:31.534340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d42c68-89f5-4b9e-9c40-b347d989efe2 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:31.593982 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d383e5-264f-457d-8f00-b656c90648c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:31.595353 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d383e5-264f-457d-8f00-b656c90648c3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:31.595637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d383e5-264f-457d-8f00-b656c90648c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:31.596172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d383e5-264f-457d-8f00-b656c90648c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:31.597255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d383e5-264f-457d-8f00-b656c90648c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:31.597655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d383e5-264f-457d-8f00-b656c90648c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:31.599675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d383e5-264f-457d-8f00-b656c90648c3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:31.608161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 463/927] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:31.619842 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 465/928] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:31.625614 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a361a31-c5bb-4c57-a807-6e31671950ea None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:31.661622 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d383e5-264f-457d-8f00-b656c90648c3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:31.737654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a361a31-c5bb-4c57-a807-6e31671950ea None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:31.778893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2810f8-40ef-421a-b8d2-092096329a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:31.779523 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2810f8-40ef-421a-b8d2-092096329a7e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:31.779794 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2810f8-40ef-421a-b8d2-092096329a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:31.780061 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2810f8-40ef-421a-b8d2-092096329a7e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:31.787820 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2810f8-40ef-421a-b8d2-092096329a7e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:31.789241 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2810f8-40ef-421a-b8d2-092096329a7e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:31.798026 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d383e5-264f-457d-8f00-b656c90648c3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:31.801829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-1c2810f8-40ef-421a-b8d2-092096329a7e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:14:31.801829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:31.801829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:31.801829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:31.801829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:31.801829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:31.801829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:31.801829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:31.801829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:31.801829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:31.801829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:31.801829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:31.801829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:31.801829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:31.801829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:31.801829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:31.802783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:31.803710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:31.803710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:31.803710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:31.803710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:31.803710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:31.803710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:31.803710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:31.803710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:31.803710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:14:31.803710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:31.803710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 464/929] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 15:14:31 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:31.815273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2810f8-40ef-421a-b8d2-092096329a7e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:31.815809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:31.820312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:31.850906 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996a749-5c2d-4914-90d9-11104202c733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:31.852122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996a749-5c2d-4914-90d9-11104202c733 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:31.852913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996a749-5c2d-4914-90d9-11104202c733 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:31.853800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5996a749-5c2d-4914-90d9-11104202c733 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:31.855439 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996a749-5c2d-4914-90d9-11104202c733 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:31.858479 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5996a749-5c2d-4914-90d9-11104202c733 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-5996a749-5c2d-4914-90d9-11104202c733 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:31.870704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:31.871698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:31.871698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:31.871698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:31.871698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:31.871698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:31.871698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:31.871698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:31.871698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:31.871698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:31.871698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:31.871698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:31.871698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:31.871698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:31.871698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:31.871698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:31.871698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:31.872529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:31.872529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:31.872529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:31.872529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:31.872529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:31.872529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:31.872529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:31.872529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:31.872529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:31.872529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:31.872529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:31.872529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:31.872529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:14:31.872529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:31.875555 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 466/930] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 15:14:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:31.881558 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996a749-5c2d-4914-90d9-11104202c733 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:31.942781 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2810f8-40ef-421a-b8d2-092096329a7e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:31.980196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d38db7-f338-46f2-a45c-274015b6c1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:31.980798 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d38db7-f338-46f2-a45c-274015b6c1f7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:31.981071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d38db7-f338-46f2-a45c-274015b6c1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:31.981341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d38db7-f338-46f2-a45c-274015b6c1f7 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:31.982066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d38db7-f338-46f2-a45c-274015b6c1f7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:31.982505 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d38db7-f338-46f2-a45c-274015b6c1f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:31.984045 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d38db7-f338-46f2-a45c-274015b6c1f7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:31.985900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5996a749-5c2d-4914-90d9-11104202c733 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:32.004580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 465/931] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 24 15:14:31 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:32.004580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d38db7-f338-46f2-a45c-274015b6c1f7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:32.025810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762dd087-72c4-44d0-a94a-97650be4940a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:32.026711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dd087-72c4-44d0-a94a-97650be4940a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:32.026995 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dd087-72c4-44d0-a94a-97650be4940a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:32.027265 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dd087-72c4-44d0-a94a-97650be4940a None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:32.027897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762dd087-72c4-44d0-a94a-97650be4940a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:32.028196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762dd087-72c4-44d0-a94a-97650be4940a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:32.029559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762dd087-72c4-44d0-a94a-97650be4940a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:32.038257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 467/932] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 15:14:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:32.038257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762dd087-72c4-44d0-a94a-97650be4940a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:32.076339 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:32.082377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:32.122551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d38db7-f338-46f2-a45c-274015b6c1f7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:32.145127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762dd087-72c4-44d0-a94a-97650be4940a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:32.159508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185c1f6-3792-4ee5-958b-63ad4ca0295e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:32.161115 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185c1f6-3792-4ee5-958b-63ad4ca0295e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:32.161400 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185c1f6-3792-4ee5-958b-63ad4ca0295e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:32.161783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d185c1f6-3792-4ee5-958b-63ad4ca0295e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:32.162406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185c1f6-3792-4ee5-958b-63ad4ca0295e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:32.163765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d185c1f6-3792-4ee5-958b-63ad4ca0295e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:32.172622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d185c1f6-3792-4ee5-958b-63ad4ca0295e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:14:32.172622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:32.172622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:32.172622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:32.172622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:32.172622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:32.172622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:32.172622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:32.172622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:14:32.172622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:32.172622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:32.172622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:32.172622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:32.172622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:32.172622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:32.172622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:32.173543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:32.174544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:32.174544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:32.174544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:32.174544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:32.174544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:32.174544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:32.174544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:32.174544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:32.174544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:32.174544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:32.174544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:32.174544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:32.174544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 15:14:32.174544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:32.174544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 466/933] 10.222.0.76 () {56 vars in 1132 bytes} [Wed Aug 24 15:14:32 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:32.175329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185c1f6-3792-4ee5-958b-63ad4ca0295e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:32.187410 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bf722-0749-43a5-a241-e6045bb716e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:32.188042 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bf722-0749-43a5-a241-e6045bb716e8 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:32.188320 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bf722-0749-43a5-a241-e6045bb716e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:32.188589 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983bf722-0749-43a5-a241-e6045bb716e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:32.199859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983bf722-0749-43a5-a241-e6045bb716e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:32.201308 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983bf722-0749-43a5-a241-e6045bb716e8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:32.216682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-983bf722-0749-43a5-a241-e6045bb716e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:32.216682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:32.216682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:32.216682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:32.216682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:32.216682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:32.216682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:32.216682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:32.216682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:32.216682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:32.216682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:32.216682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:32.216682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:32.216682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:32.216682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:32.216682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:32.217661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:32.218581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:32.218581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:32.218581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:32.218581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 468/934] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:32.219230 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bf722-0749-43a5-a241-e6045bb716e8 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:32.290268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d185c1f6-3792-4ee5-958b-63ad4ca0295e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:32.319306 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:32.324006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:32.334325 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb46f6-b647-41c9-be34-f8f6fe9835c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:32.335140 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb46f6-b647-41c9-be34-f8f6fe9835c9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:32.335479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb46f6-b647-41c9-be34-f8f6fe9835c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:32.335648 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983bf722-0749-43a5-a241-e6045bb716e8 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:32.335746 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfb46f6-b647-41c9-be34-f8f6fe9835c9 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:32.336432 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb46f6-b647-41c9-be34-f8f6fe9835c9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:32.336779 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb46f6-b647-41c9-be34-f8f6fe9835c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:32.338466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfb46f6-b647-41c9-be34-f8f6fe9835c9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:32.345347 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 467/935] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 15:14:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:32.346284 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb46f6-b647-41c9-be34-f8f6fe9835c9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:32.386498 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab33a2-b8b3-44fe-99f0-0a86525c1318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:32.387337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab33a2-b8b3-44fe-99f0-0a86525c1318 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:32.387777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab33a2-b8b3-44fe-99f0-0a86525c1318 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:32.388305 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ab33a2-b8b3-44fe-99f0-0a86525c1318 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:32.389196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab33a2-b8b3-44fe-99f0-0a86525c1318 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:32.389493 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab33a2-b8b3-44fe-99f0-0a86525c1318 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:32.390892 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ab33a2-b8b3-44fe-99f0-0a86525c1318 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:32.398396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 469/936] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 15:14:32 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:32.398396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab33a2-b8b3-44fe-99f0-0a86525c1318 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:32.460126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfb46f6-b647-41c9-be34-f8f6fe9835c9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:32.495383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0788d-51b7-4059-87ab-a19637bb8ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:32.496410 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0788d-51b7-4059-87ab-a19637bb8ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:32.496685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0788d-51b7-4059-87ab-a19637bb8ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:32.497381 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0788d-51b7-4059-87ab-a19637bb8ba4 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:32.498028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0788d-51b7-4059-87ab-a19637bb8ba4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:32.498356 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0788d-51b7-4059-87ab-a19637bb8ba4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:32.499724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0788d-51b7-4059-87ab-a19637bb8ba4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:32.517491 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 468/937] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:32.517710 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ab33a2-b8b3-44fe-99f0-0a86525c1318 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:32.517809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0788d-51b7-4059-87ab-a19637bb8ba4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:32.560306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff392e-69e5-412f-b12a-fe9638dc42b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:32.560918 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff392e-69e5-412f-b12a-fe9638dc42b2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:32.561198 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff392e-69e5-412f-b12a-fe9638dc42b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:32.561469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff392e-69e5-412f-b12a-fe9638dc42b2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:32.562172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff392e-69e5-412f-b12a-fe9638dc42b2 None admin] RBAC: Adding query filter params (interface=public, service_id=e9478dcc8a384096a606636bcbb0c7fd, region_id=RegionOne) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:32.562517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff392e-69e5-412f-b12a-fe9638dc42b2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:32.563874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff392e-69e5-412f-b12a-fe9638dc42b2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:32.569337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 470/938] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 24 15:14:32 2022] GET /identity/v3/endpoints?service_id=e9478dcc8a384096a606636bcbb0c7fd&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:32.573954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff392e-69e5-412f-b12a-fe9638dc42b2 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:32.580744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:32.597278 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:32.640255 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0788d-51b7-4059-87ab-a19637bb8ba4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:32.675297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b2b3f-ede0-4910-8b70-25be643d3b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:32.675928 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b2b3f-ede0-4910-8b70-25be643d3b82 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:32.676207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b2b3f-ede0-4910-8b70-25be643d3b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:32.676479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6b2b3f-ede0-4910-8b70-25be643d3b82 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:32.677106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b2b3f-ede0-4910-8b70-25be643d3b82 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:32.677407 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b2b3f-ede0-4910-8b70-25be643d3b82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:32.678791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6b2b3f-ede0-4910-8b70-25be643d3b82 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:32.683621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 469/939] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 15:14:32 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:32.684243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b2b3f-ede0-4910-8b70-25be643d3b82 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:32.692371 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff392e-69e5-412f-b12a-fe9638dc42b2 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:32.729636 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0684de0-4826-4bad-a71c-47b73dcaa25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:32.730310 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0684de0-4826-4bad-a71c-47b73dcaa25a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:32.730592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0684de0-4826-4bad-a71c-47b73dcaa25a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:32.730864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0684de0-4826-4bad-a71c-47b73dcaa25a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:32.738801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0684de0-4826-4bad-a71c-47b73dcaa25a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:32.740247 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0684de0-4826-4bad-a71c-47b73dcaa25a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:32.751711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-f0684de0-4826-4bad-a71c-47b73dcaa25a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:32.751711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:32.751711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:32.751711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:32.751711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:32.751711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:32.751711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:32.751711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:32.751711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:32.751711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:32.751711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:32.751711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:32.751711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:32.751711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:32.751711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:32.752795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:32.753733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:32.753733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:32.753733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:32.753733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:32.753733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:32.753733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:32.753733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:32.753733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:32.753733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 471/940] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:32.754790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0684de0-4826-4bad-a71c-47b73dcaa25a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:32.804801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6b2b3f-ede0-4910-8b70-25be643d3b82 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:32.820789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:32.845080 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580eec85-97eb-44df-a977-2b3699000e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:32.845510 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:32.845686 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580eec85-97eb-44df-a977-2b3699000e73 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:32.845958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580eec85-97eb-44df-a977-2b3699000e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:32.846242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580eec85-97eb-44df-a977-2b3699000e73 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:32.846861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580eec85-97eb-44df-a977-2b3699000e73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:32.849494 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580eec85-97eb-44df-a977-2b3699000e73 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:32.856538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 470/941] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 15:14:32 2022] GET /identity/v3/services => generated 1497 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:14:32.857354 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580eec85-97eb-44df-a977-2b3699000e73 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:32.880375 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0684de0-4826-4bad-a71c-47b73dcaa25a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:32.938031 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba329b2-1836-43c3-ab8a-537e3ef62064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:32.939539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba329b2-1836-43c3-ab8a-537e3ef62064 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:32.940018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba329b2-1836-43c3-ab8a-537e3ef62064 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:32.940710 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba329b2-1836-43c3-ab8a-537e3ef62064 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:32.942771 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba329b2-1836-43c3-ab8a-537e3ef62064 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:32.944376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba329b2-1836-43c3-ab8a-537e3ef62064 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:32.982270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 472/942] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:14:32 2022] POST /identity/v3/endpoints => generated 347 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:14:32.989147 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580eec85-97eb-44df-a977-2b3699000e73 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:33.041584 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a46473-3c6b-4c8f-997e-406ff9dd1ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:33.042244 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a46473-3c6b-4c8f-997e-406ff9dd1ab0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:33.042547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a46473-3c6b-4c8f-997e-406ff9dd1ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:33.042820 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a46473-3c6b-4c8f-997e-406ff9dd1ab0 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:33.043547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a46473-3c6b-4c8f-997e-406ff9dd1ab0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:33.043857 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a46473-3c6b-4c8f-997e-406ff9dd1ab0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:33.045338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a46473-3c6b-4c8f-997e-406ff9dd1ab0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:33.059953 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 471/943] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:33.066147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba329b2-1836-43c3-ab8a-537e3ef62064 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:33.189728 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba329b2-1836-43c3-ab8a-537e3ef62064 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:33.226518 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5aa3fc-a086-4275-8551-20a9f587931c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:33.227188 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5aa3fc-a086-4275-8551-20a9f587931c None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:33.227471 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5aa3fc-a086-4275-8551-20a9f587931c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:33.227749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5aa3fc-a086-4275-8551-20a9f587931c None admin] PATH_INFO: `/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/ad682eade4104a799b21f47aa7cdd4f5/roles/c87fe7f37cb246e8af3bc71057ec5048` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:33.230072 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5aa3fc-a086-4275-8551-20a9f587931c None admin] RBAC: Authorizing `identity:create_grant(project_id=2c010096b77f4cd3b2f32086b1ba7ea7, user_id=ad682eade4104a799b21f47aa7cdd4f5, role_id=c87fe7f37cb246e8af3bc71057ec5048)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:33.231551 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5aa3fc-a086-4275-8551-20a9f587931c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:33.234927 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:33.249958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 473/944] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:14:33 2022] PUT /identity/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/ad682eade4104a799b21f47aa7cdd4f5/roles/c87fe7f37cb246e8af3bc71057ec5048 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:14:33.491830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6ec50-47c5-4610-9eba-45ac2b116b5d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:33.492323 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6ec50-47c5-4610-9eba-45ac2b116b5d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:33.492323 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6ec50-47c5-4610-9eba-45ac2b116b5d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:33.493922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 472/945] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:33.501836 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e9d2-8e46-4f82-b9f3-dd52de19cb25 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:33.502376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e9d2-8e46-4f82-b9f3-dd52de19cb25 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:33.502576 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886e9d2-8e46-4f82-b9f3-dd52de19cb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:33.527082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e886e9d2-8e46-4f82-b9f3-dd52de19cb25 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:33.528277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e886e9d2-8e46-4f82-b9f3-dd52de19cb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:33.600345 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 474/946] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:33.871170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6ec50-47c5-4610-9eba-45ac2b116b5d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:33.905440 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:33.987212 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c6ec50-47c5-4610-9eba-45ac2b116b5d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:34.023765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed098877-af12-4bea-bdd9-3ec5924303ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:34.024731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed098877-af12-4bea-bdd9-3ec5924303ed None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:34.025195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed098877-af12-4bea-bdd9-3ec5924303ed None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:34.025619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed098877-af12-4bea-bdd9-3ec5924303ed None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:34.026844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed098877-af12-4bea-bdd9-3ec5924303ed None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:34.028515 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed098877-af12-4bea-bdd9-3ec5924303ed None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:34.031712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: WARNING keystone.server.flask.common [None req-ed098877-af12-4bea-bdd9-3ec5924303ed None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 15:14:34.052631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.sql.core [None req-ed098877-af12-4bea-bdd9-3ec5924303ed None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 15:14:34.052631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 15:14:34.052631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [parameters: {'id': 'e0a172dfd30b47e9a2715317089ce1aa', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 15:14:34.052631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81620) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 15:14:34.061075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ed098877-af12-4bea-bdd9-3ec5924303ed None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e0a172dfd30b47e9a2715317089ce1aa.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e0a172dfd30b47e9a2715317089ce1aa. Aug 24 15:14:34.061075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:34.061075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 15:14:34.061075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 15:14:34.061075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 15:14:34.061075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 15:14:34.061075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 15:14:34.061075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 15:14:34.061075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 15:14:34.061075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application conn.query(q) Aug 24 15:14:34.061075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 15:14:34.061075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 15:14:34.061075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result.read() Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 15:14:34.062074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application next(self.gen) Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application yield resource Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application next(self.gen) Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise self.value Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application session.commit() Aug 24 15:14:34.063303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.session.flush() Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application compat.raise_( Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 15:14:34.064385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 15:14:34.065418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 15:14:34.065418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 15:14:34.065418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 15:14:34.065418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 15:14:34.065418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 15:14:34.065418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 15:14:34.065418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 15:14:34.065418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 15:14:34.065418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 15:14:34.065418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 15:14:34.065418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 15:14:34.065418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 15:14:34.065418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 15:14:34.065418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 15:14:34.065418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application conn.query(q) Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application result.read() Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 15:14:34.066233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 15:14:34.067243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 15:14:34.067243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 15:14:34.067243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 15:14:34.067243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 15:14:34.067243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 15:14:34.067243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 15:14:34.067243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 15:14:34.067243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 15:14:34.067243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [parameters: {'id': 'e0a172dfd30b47e9a2715317089ce1aa', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 15:14:34.067243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 15:14:34.067243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:34.067243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:34.067243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:34.067243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:34.067243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:34.068035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:34.069029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:34.069029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 15:14:34.069029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 15:14:34.069029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:34.069029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:34.069029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 15:14:34.069029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 15:14:34.069029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e0a172dfd30b47e9a2715317089ce1aa. Aug 24 15:14:34.069029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:34.071194 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 473/947] 10.222.0.76 () {60 vars in 1140 bytes} [Wed Aug 24 15:14:33 2022] POST /identity/v3/projects => generated 262 bytes in 201 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:34.078271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886e9d2-8e46-4f82-b9f3-dd52de19cb25 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:34.191799 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886e9d2-8e46-4f82-b9f3-dd52de19cb25 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:34.218864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:34.228625 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81305b3c-3096-40e3-9e7d-efea89e960c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:34.229648 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81305b3c-3096-40e3-9e7d-efea89e960c2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:34.230072 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81305b3c-3096-40e3-9e7d-efea89e960c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:34.230685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81305b3c-3096-40e3-9e7d-efea89e960c2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:34.237922 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81305b3c-3096-40e3-9e7d-efea89e960c2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:34.239617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81305b3c-3096-40e3-9e7d-efea89e960c2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:34.251170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-81305b3c-3096-40e3-9e7d-efea89e960c2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:34.251170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:34.251170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:34.251170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:34.251170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:34.251170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:34.251170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:34.251170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:34.251170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:34.251170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:34.251170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:34.251170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:34.251170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:34.251170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:34.251170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:34.252251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:34.253314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:34.253314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:34.253314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:34.253314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:34.253314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:34.253314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:34.253314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:34.253314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:34.260513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 475/948] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:34.262212 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed098877-af12-4bea-bdd9-3ec5924303ed None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:34.366563 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:34.381589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed098877-af12-4bea-bdd9-3ec5924303ed None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:34.418849 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82722dd-9bef-4201-ad91-a79399e4641e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:34.419636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82722dd-9bef-4201-ad91-a79399e4641e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:34.419918 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82722dd-9bef-4201-ad91-a79399e4641e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:34.420190 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82722dd-9bef-4201-ad91-a79399e4641e None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:34.420857 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82722dd-9bef-4201-ad91-a79399e4641e None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:34.421160 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82722dd-9bef-4201-ad91-a79399e4641e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:34.422875 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82722dd-9bef-4201-ad91-a79399e4641e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:34.444564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 474/949] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 15:14:34 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:34.951215 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4195e-de51-41c6-b5ec-b5e35b309dce None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:34.951745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4195e-de51-41c6-b5ec-b5e35b309dce None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:34.952040 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4195e-de51-41c6-b5ec-b5e35b309dce None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:34.953727 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 476/950] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:34.960036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cbf4c-d47e-42dd-80d1-2b7e34dc49d2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:34.960512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cbf4c-d47e-42dd-80d1-2b7e34dc49d2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:34.960791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cbf4c-d47e-42dd-80d1-2b7e34dc49d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:34.979972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d92cbf4c-d47e-42dd-80d1-2b7e34dc49d2 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:34.981512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d92cbf4c-d47e-42dd-80d1-2b7e34dc49d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:35.023275 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63f294-9c06-4bb5-89be-acb23706dce0 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:35.023614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63f294-9c06-4bb5-89be-acb23706dce0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:35.023814 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae63f294-9c06-4bb5-89be-acb23706dce0 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:35.025371 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 477/951] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:35.031333 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e29c5-bb37-45a3-9668-20e6f26136cf None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:35.031611 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e29c5-bb37-45a3-9668-20e6f26136cf None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:35.031879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e29c5-bb37-45a3-9668-20e6f26136cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:35.051898 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-dd6e29c5-bb37-45a3-9668-20e6f26136cf None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:35.053081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-dd6e29c5-bb37-45a3-9668-20e6f26136cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:35.055175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 475/952] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:34 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:35.127810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 478/953] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:35.221059 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b847d-e840-4e9c-8da9-da9e0129d215 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:35.221254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b847d-e840-4e9c-8da9-da9e0129d215 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:35.221526 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b847d-e840-4e9c-8da9-da9e0129d215 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:35.223469 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 476/954] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:35.228920 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e4a46-557b-48bb-b3ab-70e23d120779 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:35.229209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e4a46-557b-48bb-b3ab-70e23d120779 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:35.229474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e4a46-557b-48bb-b3ab-70e23d120779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:35.248448 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5a9e4a46-557b-48bb-b3ab-70e23d120779 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:35.249634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5a9e4a46-557b-48bb-b3ab-70e23d120779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:35.318948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 479/955] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:35.342156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b847d-e840-4e9c-8da9-da9e0129d215 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:35.403141 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e4a46-557b-48bb-b3ab-70e23d120779 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:35.428701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:35.453804 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b847d-e840-4e9c-8da9-da9e0129d215 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:35.469421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:35.494152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a7c64-4b6f-4798-985a-6331eec28af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:35.495192 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a7c64-4b6f-4798-985a-6331eec28af0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:35.495623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a7c64-4b6f-4798-985a-6331eec28af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:35.496042 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a7c64-4b6f-4798-985a-6331eec28af0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:35.502329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a7c64-4b6f-4798-985a-6331eec28af0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:35.504158 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a7c64-4b6f-4798-985a-6331eec28af0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b75a7c64-4b6f-4798-985a-6331eec28af0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:35.514935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:35.516340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:35.516340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:35.516340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:35.516340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:35.516340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:35.516340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:35.516340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:35.516340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:35.516340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:35.516340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:35.516340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:35.516340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:35.516340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:35.516340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:35.516340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:35.516340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:35.517360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:35.517360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:35.517360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:35.517360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:35.517360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:35.517360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:35.517360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:35.517360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:35.517360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:35.517360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:35.517360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:35.517360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:35.517360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:14:35.517360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:35.520604 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 477/956] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 15:14:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:35.529986 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a7c64-4b6f-4798-985a-6331eec28af0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:35.536329 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e4a46-557b-48bb-b3ab-70e23d120779 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:35.581416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935464c6-1ddf-4b10-9420-db77ca307649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:35.582912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935464c6-1ddf-4b10-9420-db77ca307649 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:35.583190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935464c6-1ddf-4b10-9420-db77ca307649 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:35.583454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935464c6-1ddf-4b10-9420-db77ca307649 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:35.584504 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935464c6-1ddf-4b10-9420-db77ca307649 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:35.585885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935464c6-1ddf-4b10-9420-db77ca307649 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-935464c6-1ddf-4b10-9420-db77ca307649 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:35.596904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:35.598092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:35.598092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:35.598092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:35.598092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:35.598092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:35.598092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:35.598092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:35.598092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:35.598092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:35.598092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:35.598092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:35.598092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:35.598092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:35.598092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:35.598092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:35.598092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:35.599001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:35.599001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:35.599001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:35.599001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:35.599001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:35.599001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:35.599001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:35.599001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:35.599001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:35.599001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:35.599001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:35.599001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:35.599001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:14:35.599001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:35.603154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 480/957] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:35 2022] GET /identity/v3/services/compute => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:35.603714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935464c6-1ddf-4b10-9420-db77ca307649 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:35.649762 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a7c64-4b6f-4798-985a-6331eec28af0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:35.685346 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686177a-2f81-402a-82d9-bd1b69bdedd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:35.685949 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686177a-2f81-402a-82d9-bd1b69bdedd4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:35.686254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686177a-2f81-402a-82d9-bd1b69bdedd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:35.686531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2686177a-2f81-402a-82d9-bd1b69bdedd4 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:35.687155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2686177a-2f81-402a-82d9-bd1b69bdedd4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:35.687453 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2686177a-2f81-402a-82d9-bd1b69bdedd4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:35.688811 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2686177a-2f81-402a-82d9-bd1b69bdedd4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:35.694250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 478/958] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 15:14:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:35.699231 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686177a-2f81-402a-82d9-bd1b69bdedd4 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:35.709462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:35.718659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935464c6-1ddf-4b10-9420-db77ca307649 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:35.730162 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:35.754986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318e287-7506-49ed-a9e0-80a0fe76748f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:35.755529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318e287-7506-49ed-a9e0-80a0fe76748f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:35.755808 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318e287-7506-49ed-a9e0-80a0fe76748f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:35.756074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8318e287-7506-49ed-a9e0-80a0fe76748f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:35.766850 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8318e287-7506-49ed-a9e0-80a0fe76748f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:35.772556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-8318e287-7506-49ed-a9e0-80a0fe76748f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:14:35.772556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:35.772556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:35.772556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:35.772556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:35.772556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:35.772556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:35.772556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:35.772556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:35.772556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:35.772556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:35.772556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:35.772556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:35.772556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:35.772556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:35.772556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:35.773602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:35.773602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:35.773602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:35.773602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:35.773602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:35.773602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:35.773602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:35.773602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:35.773602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:35.773602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:35.773602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:35.773602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:35.773602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:35.773602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:35.773602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:35.773602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:35.774822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:35.774822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:35.774822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:35.774822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:35.774822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:35.774822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:35.774822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:35.774822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:35.774822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:35.774822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:35.774822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:14:35.774822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:35.774822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 481/959] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 24 15:14:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:35.774822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318e287-7506-49ed-a9e0-80a0fe76748f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:35.818738 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2686177a-2f81-402a-82d9-bd1b69bdedd4 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:35.858298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c6076-edba-4a66-9723-9e8c3cac59c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:35.859062 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c6076-edba-4a66-9723-9e8c3cac59c9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:35.859435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c6076-edba-4a66-9723-9e8c3cac59c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:35.859875 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055c6076-edba-4a66-9723-9e8c3cac59c9 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:35.860528 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c6076-edba-4a66-9723-9e8c3cac59c9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:35.860929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c6076-edba-4a66-9723-9e8c3cac59c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:35.862423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055c6076-edba-4a66-9723-9e8c3cac59c9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:35.873020 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 479/960] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 15:14:35 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:35.873020 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c6076-edba-4a66-9723-9e8c3cac59c9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:35.889960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8318e287-7506-49ed-a9e0-80a0fe76748f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:35.927225 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae553b-1eda-4f96-98eb-a591fce4c34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:35.927921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae553b-1eda-4f96-98eb-a591fce4c34c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:35.928257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae553b-1eda-4f96-98eb-a591fce4c34c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:35.928595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae553b-1eda-4f96-98eb-a591fce4c34c None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:35.929481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae553b-1eda-4f96-98eb-a591fce4c34c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:35.930163 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae553b-1eda-4f96-98eb-a591fce4c34c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:35.931507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae553b-1eda-4f96-98eb-a591fce4c34c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:35.942803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 482/961] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 15:14:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:35.942803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae553b-1eda-4f96-98eb-a591fce4c34c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:35.959533 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:35.974491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:35.996364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055c6076-edba-4a66-9723-9e8c3cac59c9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:36.034779 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99992f-f703-4f00-ba5d-115c1fd16645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:36.035326 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99992f-f703-4f00-ba5d-115c1fd16645 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:36.035607 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99992f-f703-4f00-ba5d-115c1fd16645 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:36.035877 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99992f-f703-4f00-ba5d-115c1fd16645 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:36.036542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99992f-f703-4f00-ba5d-115c1fd16645 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:36.036845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99992f-f703-4f00-ba5d-115c1fd16645 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:36.038209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99992f-f703-4f00-ba5d-115c1fd16645 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:36.045047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae553b-1eda-4f96-98eb-a591fce4c34c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:36.050269 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 480/962] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 24 15:14:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:36.050269 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99992f-f703-4f00-ba5d-115c1fd16645 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:36.084040 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262f9b4-424d-4ff3-b66e-7d5eb374ef18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:36.084638 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262f9b4-424d-4ff3-b66e-7d5eb374ef18 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:36.084915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262f9b4-424d-4ff3-b66e-7d5eb374ef18 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:36.085178 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9262f9b4-424d-4ff3-b66e-7d5eb374ef18 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:36.085801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262f9b4-424d-4ff3-b66e-7d5eb374ef18 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:36.086100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262f9b4-424d-4ff3-b66e-7d5eb374ef18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:36.087501 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9262f9b4-424d-4ff3-b66e-7d5eb374ef18 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:36.098630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 483/963] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 15:14:35 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:36.100422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262f9b4-424d-4ff3-b66e-7d5eb374ef18 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:36.175102 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99992f-f703-4f00-ba5d-115c1fd16645 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:36.218588 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:36.230083 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85ca7a-40a6-400d-a658-40455e5a3f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:36.230733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:36.231799 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85ca7a-40a6-400d-a658-40455e5a3f23 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:36.233101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85ca7a-40a6-400d-a658-40455e5a3f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:36.233619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a85ca7a-40a6-400d-a658-40455e5a3f23 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:36.234831 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85ca7a-40a6-400d-a658-40455e5a3f23 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:36.236441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a85ca7a-40a6-400d-a658-40455e5a3f23 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:36.237172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9262f9b4-424d-4ff3-b66e-7d5eb374ef18 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:36.265894 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 481/964] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:14:36 2022] POST /identity/v3/endpoints => generated 349 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:14:36.292884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c0a10d-092b-4bf1-9e88-8d950144213b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:36.293443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0a10d-092b-4bf1-9e88-8d950144213b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:36.293719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0a10d-092b-4bf1-9e88-8d950144213b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:36.293984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c0a10d-092b-4bf1-9e88-8d950144213b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:36.306832 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0a10d-092b-4bf1-9e88-8d950144213b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:36.308363 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c0a10d-092b-4bf1-9e88-8d950144213b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:36.325057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-43c0a10d-092b-4bf1-9e88-8d950144213b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:36.325057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:36.325057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:36.325057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:36.325057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:36.325057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:36.325057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:36.325057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:36.325057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:36.325057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:36.325057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:36.325057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:36.325057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:36.325057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:36.325057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:36.325057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:36.326080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:36.327111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:36.327111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:36.327111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:36.327111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 484/965] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:36.333273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85ca7a-40a6-400d-a658-40455e5a3f23 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:36.454879 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a85ca7a-40a6-400d-a658-40455e5a3f23 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:36.481702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a79013-5ad0-4039-927f-ccf7a9103b9f None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:36.482455 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a79013-5ad0-4039-927f-ccf7a9103b9f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:36.482973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a79013-5ad0-4039-927f-ccf7a9103b9f None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:36.484799 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 485/966] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:14:36.490997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff8c59-42c4-4967-a134-34e20b479adc None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:36.491593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff8c59-42c4-4967-a134-34e20b479adc None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:36.492082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ff8c59-42c4-4967-a134-34e20b479adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:36.493845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9ebdf-88bc-4432-a71d-7ec561db6fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:36.494479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9ebdf-88bc-4432-a71d-7ec561db6fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:36.494755 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9ebdf-88bc-4432-a71d-7ec561db6fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:36.495023 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce9ebdf-88bc-4432-a71d-7ec561db6fc1 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:36.495663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9ebdf-88bc-4432-a71d-7ec561db6fc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:36.495962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9ebdf-88bc-4432-a71d-7ec561db6fc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:36.500170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce9ebdf-88bc-4432-a71d-7ec561db6fc1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:36.511156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 482/967] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:36.519305 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9ebdf-88bc-4432-a71d-7ec561db6fc1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:36.519889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-46ff8c59-42c4-4967-a134-34e20b479adc None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:36.521376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-46ff8c59-42c4-4967-a134-34e20b479adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:36.585206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:36.626252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 486/968] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:36 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:36.647047 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce9ebdf-88bc-4432-a71d-7ec561db6fc1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:36.683148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c6cfd-ccc2-47e2-9310-fa9b22e21b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:36.683775 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c6cfd-ccc2-47e2-9310-fa9b22e21b65 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:36.684051 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c6cfd-ccc2-47e2-9310-fa9b22e21b65 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:36.684323 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c6cfd-ccc2-47e2-9310-fa9b22e21b65 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:36.692290 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c6cfd-ccc2-47e2-9310-fa9b22e21b65 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:36.693771 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c6cfd-ccc2-47e2-9310-fa9b22e21b65 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:36.705542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8f4c6cfd-ccc2-47e2-9310-fa9b22e21b65 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:14:36.705542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:36.705542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:36.705542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:36.705542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:36.705542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:36.705542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:36.705542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:36.705542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:36.705542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:36.705542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:36.705542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:36.705542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:36.705542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:36.705542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:36.705542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:36.706612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:36.707655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:36.707655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:36.707655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:36.707655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:36.707655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:36.707655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:36.707655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:36.707655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:36.707655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 15:14:36.707655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:36.707655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 483/969] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 15:14:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:36.712518 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff8c59-42c4-4967-a134-34e20b479adc None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:36.797538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:36.822346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ff8c59-42c4-4967-a134-34e20b479adc None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:36.860047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202dd594-2b5d-4937-a497-ee068fdcfb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:36.860587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dd594-2b5d-4937-a497-ee068fdcfb06 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:36.860867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dd594-2b5d-4937-a497-ee068fdcfb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:36.861141 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202dd594-2b5d-4937-a497-ee068fdcfb06 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:36.861878 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202dd594-2b5d-4937-a497-ee068fdcfb06 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:36.862183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202dd594-2b5d-4937-a497-ee068fdcfb06 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:36.863678 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202dd594-2b5d-4937-a497-ee068fdcfb06 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:36.882089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 487/970] 10.222.0.76 () {56 vars in 1149 bytes} [Wed Aug 24 15:14:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:36.887899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c6cfd-ccc2-47e2-9310-fa9b22e21b65 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:36.906457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202dd594-2b5d-4937-a497-ee068fdcfb06 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:37.009092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202dd594-2b5d-4937-a497-ee068fdcfb06 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:37.016631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c6cfd-ccc2-47e2-9310-fa9b22e21b65 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:37.030457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:37.054307 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22a8a2-d6a3-4512-b40a-ac7ef992193d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:37.055285 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22a8a2-d6a3-4512-b40a-ac7ef992193d None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:37.055768 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22a8a2-d6a3-4512-b40a-ac7ef992193d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:37.056239 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec22a8a2-d6a3-4512-b40a-ac7ef992193d None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:37.057081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22a8a2-d6a3-4512-b40a-ac7ef992193d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:37.058840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec22a8a2-d6a3-4512-b40a-ac7ef992193d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:37.061684 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: WARNING keystone.server.flask.common [None req-ec22a8a2-d6a3-4512-b40a-ac7ef992193d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 15:14:37.064348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2eae07-55d6-4326-b88c-c955d50cdbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:37.064964 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2eae07-55d6-4326-b88c-c955d50cdbac None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:37.065243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2eae07-55d6-4326-b88c-c955d50cdbac None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:37.065517 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2eae07-55d6-4326-b88c-c955d50cdbac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:37.074762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:37.076288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.sql.core [None req-ec22a8a2-d6a3-4512-b40a-ac7ef992193d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 15:14:37.076288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 15:14:37.076288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [parameters: {'user_id': '273b743b371e4d3fa863ffe26d212319', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 15:14:37.076288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81619) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 15:14:37.094210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ec22a8a2-d6a3-4512-b40a-ac7ef992193d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 15:14:37.094210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:37.094210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 15:14:37.094210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 15:14:37.094210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 15:14:37.094210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 15:14:37.094210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 15:14:37.094210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 15:14:37.094210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 15:14:37.094210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application conn.query(q) Aug 24 15:14:37.094210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 15:14:37.094210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 15:14:37.094210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 15:14:37.094210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application result.read() Aug 24 15:14:37.094210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 15:14:37.095194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application next(self.gen) Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application yield resource Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application next(self.gen) Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise self.value Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application session.commit() Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 15:14:37.096156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self.session.flush() Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application compat.raise_( Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 15:14:37.097044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 15:14:37.098552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 15:14:37.098552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 15:14:37.098552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 15:14:37.098552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 15:14:37.098552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 15:14:37.098552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 15:14:37.098552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 15:14:37.098552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 15:14:37.098552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 15:14:37.098552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 15:14:37.098552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 15:14:37.098552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 15:14:37.098552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 15:14:37.098552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 15:14:37.098552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 15:14:37.098552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception Aug 24 15:14:37.099421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 15:14:37.099421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 15:14:37.099421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 15:14:37.099421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 15:14:37.099421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 15:14:37.099421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 15:14:37.099421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 15:14:37.099421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application conn.query(q) Aug 24 15:14:37.099421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 15:14:37.099421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 15:14:37.099421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 15:14:37.099421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application result.read() Aug 24 15:14:37.099421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 15:14:37.099421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 15:14:37.099421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 15:14:37.099421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 15:14:37.100257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 15:14:37.100257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 15:14:37.100257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 15:14:37.100257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 15:14:37.100257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 15:14:37.100257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 15:14:37.100257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [parameters: {'user_id': '273b743b371e4d3fa863ffe26d212319', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 15:14:37.100257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 15:14:37.100257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:37.100257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:37.100257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:37.100257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:37.100257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:37.100257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:37.100257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 15:14:37.101050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 15:14:37.103644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 15:14:37.103644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 15:14:37.103644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 15:14:37.103644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 15:14:37.103644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 15:14:37.103644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:37.110884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 488/971] 10.222.0.76 () {60 vars in 1131 bytes} [Wed Aug 24 15:14:36 2022] POST /identity/v3/users => generated 160 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:37.112335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22a8a2-d6a3-4512-b40a-ac7ef992193d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:37.142300 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2eae07-55d6-4326-b88c-c955d50cdbac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:37.143806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2eae07-55d6-4326-b88c-c955d50cdbac None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:37.238793 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec22a8a2-d6a3-4512-b40a-ac7ef992193d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:37.279197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 484/972] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:14:36 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:14:37.285017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2eae07-55d6-4326-b88c-c955d50cdbac None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:37.287032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9dceb-016b-46e9-83e0-9f6802a8141c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:37.288116 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9dceb-016b-46e9-83e0-9f6802a8141c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:37.288536 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9dceb-016b-46e9-83e0-9f6802a8141c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:37.288958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d9dceb-016b-46e9-83e0-9f6802a8141c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:37.296724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:37.298272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9dceb-016b-46e9-83e0-9f6802a8141c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:37.301456 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d9dceb-016b-46e9-83e0-9f6802a8141c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:37.314399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-d7d9dceb-016b-46e9-83e0-9f6802a8141c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 15:14:37.314399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:37.314399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:37.314399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:37.314399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:37.314399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:37.314399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:37.314399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:37.314399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:37.314399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:37.314399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:37.314399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:37.314399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:37.314399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:37.314399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:37.314399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:37.315404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:37.316867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:37.316867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:37.316867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:37.316867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:37.316867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:37.316867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:37.316867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:37.316867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:37.316867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 15:14:37.316867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:37.321144 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 489/973] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:14:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:37.322278 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9dceb-016b-46e9-83e0-9f6802a8141c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:37.392302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:37.412483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2eae07-55d6-4326-b88c-c955d50cdbac None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:37.451281 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512f911-9114-4329-a106-d8bd852480ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:37.451913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512f911-9114-4329-a106-d8bd852480ff None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:37.452223 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512f911-9114-4329-a106-d8bd852480ff None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:37.452497 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0512f911-9114-4329-a106-d8bd852480ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:37.457444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d9dceb-016b-46e9-83e0-9f6802a8141c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:37.463765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512f911-9114-4329-a106-d8bd852480ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:37.465307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0512f911-9114-4329-a106-d8bd852480ff None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:37.480552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0512f911-9114-4329-a106-d8bd852480ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:37.480552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:37.480552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:37.480552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:37.480552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:37.480552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:37.480552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:37.480552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:37.480552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:37.480552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:37.480552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:37.480552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:37.480552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:37.480552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:37.480552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:37.480552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:37.481522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:37.482441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:37.482441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:37.482441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:37.482441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 485/974] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:37.488569 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512f911-9114-4329-a106-d8bd852480ff None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:37.509941 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ee226-1585-4ab7-b498-b734c1845bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:37.511193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ee226-1585-4ab7-b498-b734c1845bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:37.511865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ee226-1585-4ab7-b498-b734c1845bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:37.512290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87ee226-1585-4ab7-b498-b734c1845bf7 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:37.513326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ee226-1585-4ab7-b498-b734c1845bf7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:37.513796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ee226-1585-4ab7-b498-b734c1845bf7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:37.515691 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87ee226-1585-4ab7-b498-b734c1845bf7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:37.574979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 490/975] 10.222.0.76 () {56 vars in 1101 bytes} [Wed Aug 24 15:14:37 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:37.579858 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:37.614352 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0512f911-9114-4329-a106-d8bd852480ff None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:37.654884 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1d36b-a948-4581-b851-0ac711231ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:37.655740 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1d36b-a948-4581-b851-0ac711231ecd None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:37.656171 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1d36b-a948-4581-b851-0ac711231ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:37.656589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1d36b-a948-4581-b851-0ac711231ecd None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:37.657411 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1d36b-a948-4581-b851-0ac711231ecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:37.657869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1d36b-a948-4581-b851-0ac711231ecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:37.659460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e1d36b-a948-4581-b851-0ac711231ecd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:37.673726 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 486/976] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:37.680263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ee226-1585-4ab7-b498-b734c1845bf7 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:37.792517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87ee226-1585-4ab7-b498-b734c1845bf7 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:37.810774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:37.833415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48332953-a735-4ff5-9dbc-2032fe7585ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:37.834073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48332953-a735-4ff5-9dbc-2032fe7585ea None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:37.834369 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48332953-a735-4ff5-9dbc-2032fe7585ea None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:37.834642 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48332953-a735-4ff5-9dbc-2032fe7585ea None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:37.842499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48332953-a735-4ff5-9dbc-2032fe7585ea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:37.843966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48332953-a735-4ff5-9dbc-2032fe7585ea None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:37.856579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-48332953-a735-4ff5-9dbc-2032fe7585ea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:37.856579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:37.856579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:37.856579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:37.856579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:37.856579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:37.856579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:37.856579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:37.856579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:37.856579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:37.856579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:37.856579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:37.856579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:37.856579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:37.856579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:37.857655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:37.858621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:37.858621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:37.858621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:37.858621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:37.858621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:37.858621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:37.858621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:37.858621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:37.858621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 491/977] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:37.864142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1d36b-a948-4581-b851-0ac711231ecd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:37.968380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1d36b-a948-4581-b851-0ac711231ecd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:37.989736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:38.006499 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000fd161-a74c-44c0-a68d-93bea41c7861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:38.007183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fd161-a74c-44c0-a68d-93bea41c7861 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:38.007466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fd161-a74c-44c0-a68d-93bea41c7861 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:38.007851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000fd161-a74c-44c0-a68d-93bea41c7861 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:38.008952 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fd161-a74c-44c0-a68d-93bea41c7861 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:38.009490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fd161-a74c-44c0-a68d-93bea41c7861 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:38.012009 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000fd161-a74c-44c0-a68d-93bea41c7861 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:38.036564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 487/978] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:38.037853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48332953-a735-4ff5-9dbc-2032fe7585ea None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:38.113512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14bf08-5b0e-4009-b736-56bb1baa76cc None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:38.113861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14bf08-5b0e-4009-b736-56bb1baa76cc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:38.114053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14bf08-5b0e-4009-b736-56bb1baa76cc None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:38.115617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 488/979] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:14:38.120384 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b048a-a907-42b3-b539-f10d690e8186 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:38.120669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b048a-a907-42b3-b539-f10d690e8186 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:38.120939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4b048a-a907-42b3-b539-f10d690e8186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:38.139000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-bf4b048a-a907-42b3-b539-f10d690e8186 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:38.140180 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bf4b048a-a907-42b3-b539-f10d690e8186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:38.141865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48332953-a735-4ff5-9dbc-2032fe7585ea None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:38.179004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619f4056-950b-403f-96ee-054489e9737a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:38.179912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f4056-950b-403f-96ee-054489e9737a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:38.180192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f4056-950b-403f-96ee-054489e9737a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:38.180462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f4056-950b-403f-96ee-054489e9737a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:38.181195 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f4056-950b-403f-96ee-054489e9737a None admin] RBAC: Adding query filter params (role.id=c87fe7f37cb246e8af3bc71057ec5048, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=ad682eade4104a799b21f47aa7cdd4f5) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:38.181525 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f4056-950b-403f-96ee-054489e9737a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:38.183029 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619f4056-950b-403f-96ee-054489e9737a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:38.190558 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 492/980] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:14:38 2022] GET /identity/v3/role_assignments?user.id=ad682eade4104a799b21f47aa7cdd4f5&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=c87fe7f37cb246e8af3bc71057ec5048 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:38.222536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 489/981] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:38.309832 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8329b-9743-4030-ae93-4e0972e0c952 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:38.310180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8329b-9743-4030-ae93-4e0972e0c952 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:38.310409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e8329b-9743-4030-ae93-4e0972e0c952 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:38.311941 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 493/982] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:38.318111 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f65db-37b4-44ce-869e-192e227d975b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:38.318447 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f65db-37b4-44ce-869e-192e227d975b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:38.318682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7f65db-37b4-44ce-869e-192e227d975b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:38.338550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cb7f65db-37b4-44ce-869e-192e227d975b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:38.339876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cb7f65db-37b4-44ce-869e-192e227d975b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:38.421349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 490/983] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:38.496040 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8329b-9743-4030-ae93-4e0972e0c952 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:38.549402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:38.624717 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e8329b-9743-4030-ae93-4e0972e0c952 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:38.668557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488d92b-a52b-4205-a97f-b80d05ec3da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:38.669182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488d92b-a52b-4205-a97f-b80d05ec3da0 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:38.669483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488d92b-a52b-4205-a97f-b80d05ec3da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:38.669752 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2488d92b-a52b-4205-a97f-b80d05ec3da0 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:38.670370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488d92b-a52b-4205-a97f-b80d05ec3da0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:38.671723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2488d92b-a52b-4205-a97f-b80d05ec3da0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:38.688370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 494/984] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 15:14:38 2022] POST /identity/v3/services => generated 242 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:14:38.704260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f65db-37b4-44ce-869e-192e227d975b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:38.754825 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:38.826814 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7f65db-37b4-44ce-869e-192e227d975b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:38.884519 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de197a6e-580a-4400-87cf-dead70d2fce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:38.885191 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de197a6e-580a-4400-87cf-dead70d2fce6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:38.885468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de197a6e-580a-4400-87cf-dead70d2fce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:38.885731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de197a6e-580a-4400-87cf-dead70d2fce6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:38.886347 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de197a6e-580a-4400-87cf-dead70d2fce6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:38.887728 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de197a6e-580a-4400-87cf-dead70d2fce6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-de197a6e-580a-4400-87cf-dead70d2fce6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:38.898722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:38.899791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:38.899791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:38.899791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:38.899791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:38.899791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:38.899791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:38.899791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:38.899791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:38.899791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:38.899791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:38.899791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:38.899791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:38.899791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:38.899791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:38.899791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:38.899791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:38.900618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:38.900618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:38.900618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:38.900618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:38.900618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:38.900618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:38.900618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:38.900618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:38.900618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:38.900618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:38.900618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:38.900618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:38.900618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:14:38.900618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:38.900618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 491/985] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 15:14:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:38.908805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488d92b-a52b-4205-a97f-b80d05ec3da0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:39.008238 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:39.013391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2488d92b-a52b-4205-a97f-b80d05ec3da0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:39.057785 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c9065-a540-4b79-89dd-eaee3f0eac49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:39.058820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c9065-a540-4b79-89dd-eaee3f0eac49 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:39.059032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c9065-a540-4b79-89dd-eaee3f0eac49 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:39.059302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c9065-a540-4b79-89dd-eaee3f0eac49 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:39.059934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c9065-a540-4b79-89dd-eaee3f0eac49 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:39.060231 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c9065-a540-4b79-89dd-eaee3f0eac49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:39.061612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6c9065-a540-4b79-89dd-eaee3f0eac49 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:39.066986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 495/986] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 15:14:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:39.072784 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de197a6e-580a-4400-87cf-dead70d2fce6 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:39.173289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:39.177494 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de197a6e-580a-4400-87cf-dead70d2fce6 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:39.213571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af355a07-9275-4c01-a886-5d7c97b35e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:39.214179 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af355a07-9275-4c01-a886-5d7c97b35e00 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:39.214481 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af355a07-9275-4c01-a886-5d7c97b35e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:39.214749 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af355a07-9275-4c01-a886-5d7c97b35e00 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:39.215375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af355a07-9275-4c01-a886-5d7c97b35e00 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:39.215671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af355a07-9275-4c01-a886-5d7c97b35e00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:39.217017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af355a07-9275-4c01-a886-5d7c97b35e00 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:39.222188 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 492/987] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 15:14:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:39.228683 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c9065-a540-4b79-89dd-eaee3f0eac49 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:39.339974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c9065-a540-4b79-89dd-eaee3f0eac49 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:39.377682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec62677-1c74-4fa0-a36e-3879e4f1a305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:39.378327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec62677-1c74-4fa0-a36e-3879e4f1a305 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:39.378638 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec62677-1c74-4fa0-a36e-3879e4f1a305 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:39.378910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec62677-1c74-4fa0-a36e-3879e4f1a305 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:39.379617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec62677-1c74-4fa0-a36e-3879e4f1a305 None admin] RBAC: Adding query filter params (interface=admin, service_id=e9478dcc8a384096a606636bcbb0c7fd, region_id=RegionOne) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:39.379925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec62677-1c74-4fa0-a36e-3879e4f1a305 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:39.381289 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec62677-1c74-4fa0-a36e-3879e4f1a305 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:39.387291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 496/988] 10.222.0.76 () {56 vars in 1249 bytes} [Wed Aug 24 15:14:39 2022] GET /identity/v3/endpoints?service_id=e9478dcc8a384096a606636bcbb0c7fd&interface=admin®ion_id=RegionOne => generated 193 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:39.392692 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af355a07-9275-4c01-a886-5d7c97b35e00 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:39.496381 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af355a07-9275-4c01-a886-5d7c97b35e00 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:39.539228 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65505dfc-be96-4f47-a643-e66b76679a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:39.539841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65505dfc-be96-4f47-a643-e66b76679a6d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:39.540122 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65505dfc-be96-4f47-a643-e66b76679a6d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:39.540396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65505dfc-be96-4f47-a643-e66b76679a6d None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:39.540999 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65505dfc-be96-4f47-a643-e66b76679a6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:39.542394 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65505dfc-be96-4f47-a643-e66b76679a6d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:39.549198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 493/989] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 15:14:39 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:14:40.184312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a14a1-d689-4c55-a75c-57cc7faaa3d2 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:40.184839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a14a1-d689-4c55-a75c-57cc7faaa3d2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:40.184839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281a14a1-d689-4c55-a75c-57cc7faaa3d2 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:40.186345 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 497/990] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:40.192087 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51b689-eced-4688-b4d8-47de0b5e3e90 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:40.192476 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51b689-eced-4688-b4d8-47de0b5e3e90 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:40.192652 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a51b689-eced-4688-b4d8-47de0b5e3e90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:40.211193 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3a51b689-eced-4688-b4d8-47de0b5e3e90 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:40.212379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3a51b689-eced-4688-b4d8-47de0b5e3e90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:40.236576 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:40.282723 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 494/991] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:40 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:40.546267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a14a1-d689-4c55-a75c-57cc7faaa3d2 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:40.575207 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:40.655135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281a14a1-d689-4c55-a75c-57cc7faaa3d2 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:40.675074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacd3e2-58ff-421a-bee0-345fccb12e71 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:40.675973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacd3e2-58ff-421a-bee0-345fccb12e71 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:40.676210 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacd3e2-58ff-421a-bee0-345fccb12e71 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:40.677689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 495/992] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:14:40.682486 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe20a5-b7d1-4b41-bb6c-c030494ae298 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:40.682782 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe20a5-b7d1-4b41-bb6c-c030494ae298 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:40.683045 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe20a5-b7d1-4b41-bb6c-c030494ae298 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:40.693096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d63e2c-434c-4da6-a8bb-911ff92fee0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:40.693733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d63e2c-434c-4da6-a8bb-911ff92fee0d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:40.694012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d63e2c-434c-4da6-a8bb-911ff92fee0d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:40.694308 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d63e2c-434c-4da6-a8bb-911ff92fee0d None admin] PATH_INFO: `/v3/services/image` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:40.694894 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d63e2c-434c-4da6-a8bb-911ff92fee0d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:40.696704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d63e2c-434c-4da6-a8bb-911ff92fee0d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:40.701174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-eafe20a5-b7d1-4b41-bb6c-c030494ae298 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:40.702369 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-eafe20a5-b7d1-4b41-bb6c-c030494ae298 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-a0d63e2c-434c-4da6-a8bb-911ff92fee0d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:40.705465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:40.706673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:40.706673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:40.706673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:40.706673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:40.706673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:40.706673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:40.706673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:40.706673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:40.706673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:40.706673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:40.706673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:40.706673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:40.706673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:40.706673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:40.706673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:40.706673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:40.707492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:40.707492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:40.707492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:40.707492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:40.707492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:40.707492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:40.707492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:40.707492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:40.707492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:40.707492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:40.707492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:40.707492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:40.707492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:14:40.707492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:40.707492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 498/993] 10.222.0.76 () {56 vars in 1105 bytes} [Wed Aug 24 15:14:40 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:40.712434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d63e2c-434c-4da6-a8bb-911ff92fee0d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:40.781744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 496/994] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:40 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:40.811971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:40.822942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d63e2c-434c-4da6-a8bb-911ff92fee0d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:40.860481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f2114-3638-4cdc-9417-7c39fd8c1b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:40.861138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f2114-3638-4cdc-9417-7c39fd8c1b82 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:40.861419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f2114-3638-4cdc-9417-7c39fd8c1b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:40.861685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174f2114-3638-4cdc-9417-7c39fd8c1b82 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:40.862334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f2114-3638-4cdc-9417-7c39fd8c1b82 None admin] RBAC: Adding query filter params (name=image) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:40.862636 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f2114-3638-4cdc-9417-7c39fd8c1b82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:40.864031 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174f2114-3638-4cdc-9417-7c39fd8c1b82 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:40.874041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 499/995] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:40 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:40.875693 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe20a5-b7d1-4b41-bb6c-c030494ae298 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:40.967206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:40.980863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafe20a5-b7d1-4b41-bb6c-c030494ae298 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:41.016732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ad50d-9575-4be5-8ad9-4c239e61557c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:41.017341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ad50d-9575-4be5-8ad9-4c239e61557c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:41.017620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ad50d-9575-4be5-8ad9-4c239e61557c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:41.017890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870ad50d-9575-4be5-8ad9-4c239e61557c None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:41.018546 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ad50d-9575-4be5-8ad9-4c239e61557c None admin] RBAC: Adding query filter params (type=image) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:41.018847 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ad50d-9575-4be5-8ad9-4c239e61557c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:41.020371 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870ad50d-9575-4be5-8ad9-4c239e61557c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:41.030628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 497/996] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:40 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:41.055385 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f2114-3638-4cdc-9417-7c39fd8c1b82 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:41.182076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174f2114-3638-4cdc-9417-7c39fd8c1b82 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:41.230980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26964477-94b0-4ad9-b148-bf2d6f081a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:41.231995 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26964477-94b0-4ad9-b148-bf2d6f081a4c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:41.231995 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26964477-94b0-4ad9-b148-bf2d6f081a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:41.232243 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26964477-94b0-4ad9-b148-bf2d6f081a4c None admin] PATH_INFO: `/v3/services/compute` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:41.232728 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26964477-94b0-4ad9-b148-bf2d6f081a4c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:41.234108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26964477-94b0-4ad9-b148-bf2d6f081a4c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-26964477-94b0-4ad9-b148-bf2d6f081a4c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:41.244530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:41.250744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:41.250744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:41.250744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:41.250744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:41.250744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:41.250744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:41.250744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:41.250744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:41.250744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:41.250744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:41.250744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:41.250744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:41.250744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:41.250744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:41.250744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:41.250744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:41.251612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:41.251612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:41.251612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:41.251612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:41.251612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:41.251612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:41.251612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:41.251612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:41.251612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:41.251612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:41.251612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:41.251612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:41.251612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:14:41.251612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:41.251612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 500/997] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:41.256146 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ad50d-9575-4be5-8ad9-4c239e61557c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:41.406686 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870ad50d-9575-4be5-8ad9-4c239e61557c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:41.449851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef92f6-cb89-4026-bfb1-0626184e1552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:41.450497 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef92f6-cb89-4026-bfb1-0626184e1552 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:41.450779 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef92f6-cb89-4026-bfb1-0626184e1552 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:41.451052 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef92f6-cb89-4026-bfb1-0626184e1552 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:41.451685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef92f6-cb89-4026-bfb1-0626184e1552 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:41.451989 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef92f6-cb89-4026-bfb1-0626184e1552 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:41.453374 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef92f6-cb89-4026-bfb1-0626184e1552 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:41.459769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 498/998] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 15:14:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:41.465828 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26964477-94b0-4ad9-b148-bf2d6f081a4c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:41.491567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:41.500186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d4874-e89b-4f3e-b61e-699afba189df None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:41.500186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d4874-e89b-4f3e-b61e-699afba189df None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:41.500186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5d4874-e89b-4f3e-b61e-699afba189df None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:41.500525 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 499/999] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:41.506006 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f54264-d69c-435f-8167-7cf4a19ff358 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:41.506333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f54264-d69c-435f-8167-7cf4a19ff358 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:41.506610 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f54264-d69c-435f-8167-7cf4a19ff358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:41.534446 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e9f54264-d69c-435f-8167-7cf4a19ff358 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:41.535744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e9f54264-d69c-435f-8167-7cf4a19ff358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:41.547958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:41.617185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26964477-94b0-4ad9-b148-bf2d6f081a4c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:41.618404 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 500/1000] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:41.674624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cbc32-ad4c-46ba-bd49-f4ff7fe47c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:41.675763 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cbc32-ad4c-46ba-bd49-f4ff7fe47c5f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:41.676219 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cbc32-ad4c-46ba-bd49-f4ff7fe47c5f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:41.676685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cbc32-ad4c-46ba-bd49-f4ff7fe47c5f None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:41.677772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cbc32-ad4c-46ba-bd49-f4ff7fe47c5f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:41.678314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cbc32-ad4c-46ba-bd49-f4ff7fe47c5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:41.680804 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cbc32-ad4c-46ba-bd49-f4ff7fe47c5f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:41.695295 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 501/1001] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 15:14:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:41.697004 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f54264-d69c-435f-8167-7cf4a19ff358 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:41.830876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f54264-d69c-435f-8167-7cf4a19ff358 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:41.892746 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cbc32-ad4c-46ba-bd49-f4ff7fe47c5f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:41.893846 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef6dd9-a428-42f1-be16-5843fca2bb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:41.894731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef6dd9-a428-42f1-be16-5843fca2bb11 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:41.895004 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef6dd9-a428-42f1-be16-5843fca2bb11 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:41.895354 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef6dd9-a428-42f1-be16-5843fca2bb11 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:41.896012 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef6dd9-a428-42f1-be16-5843fca2bb11 None admin] RBAC: Adding query filter params (interface=public, service_id=64d494a940454d7d84d8d1aeb6d9ead8, region_id=RegionOne) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:41.896319 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef6dd9-a428-42f1-be16-5843fca2bb11 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:41.897688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef6dd9-a428-42f1-be16-5843fca2bb11 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:41.913036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 501/1002] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 24 15:14:41 2022] GET /identity/v3/endpoints?service_id=64d494a940454d7d84d8d1aeb6d9ead8&interface=public®ion_id=RegionOne => generated 194 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:41.914319 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef6dd9-a428-42f1-be16-5843fca2bb11 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:41.935185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:42.000102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:42.020803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cbc32-ad4c-46ba-bd49-f4ff7fe47c5f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:42.046273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef6dd9-a428-42f1-be16-5843fca2bb11 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:42.060697 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5e86d-5eb7-4c50-83f4-6e26c48b020f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:42.061342 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5e86d-5eb7-4c50-83f4-6e26c48b020f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:42.061624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5e86d-5eb7-4c50-83f4-6e26c48b020f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:42.061897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be5e86d-5eb7-4c50-83f4-6e26c48b020f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:42.062524 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5e86d-5eb7-4c50-83f4-6e26c48b020f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:42.063900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be5e86d-5eb7-4c50-83f4-6e26c48b020f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-3be5e86d-5eb7-4c50-83f4-6e26c48b020f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:42.073314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:42.076236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:42.076236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:42.076236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:42.076236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:42.076236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:42.076236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:42.076236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:42.076236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:42.076236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:42.076236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:42.076236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:42.076236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:42.076236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:42.076236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:42.076236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:42.076236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:42.077120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:42.077120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:42.077120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:42.077120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:42.077120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:42.077120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:42.077120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:42.077120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:42.077120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:42.077120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:42.077120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:42.077120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:42.077120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 15:14:42.077120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:42.077120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 502/1003] 10.222.0.76 () {56 vars in 1126 bytes} [Wed Aug 24 15:14:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:42.080417 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5e86d-5eb7-4c50-83f4-6e26c48b020f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:42.089623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2206bf3-6486-4ab1-9e87-8aea1319a409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:42.090439 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2206bf3-6486-4ab1-9e87-8aea1319a409 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:42.091214 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2206bf3-6486-4ab1-9e87-8aea1319a409 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:42.091214 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2206bf3-6486-4ab1-9e87-8aea1319a409 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:42.091900 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2206bf3-6486-4ab1-9e87-8aea1319a409 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:42.093372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2206bf3-6486-4ab1-9e87-8aea1319a409 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:42.101045 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 502/1004] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 15:14:41 2022] GET /identity/v3/services => generated 1727 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:14:42.200771 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be5e86d-5eb7-4c50-83f4-6e26c48b020f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:42.239783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b9898-6aa8-4fb8-ae9a-ef79c8e9a0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:42.240449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b9898-6aa8-4fb8-ae9a-ef79c8e9a0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:42.240694 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b9898-6aa8-4fb8-ae9a-ef79c8e9a0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:42.240937 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125b9898-6aa8-4fb8-ae9a-ef79c8e9a0b2 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:42.241569 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b9898-6aa8-4fb8-ae9a-ef79c8e9a0b2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:42.241940 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b9898-6aa8-4fb8-ae9a-ef79c8e9a0b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:42.243349 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125b9898-6aa8-4fb8-ae9a-ef79c8e9a0b2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:42.249882 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 503/1005] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 15:14:42 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:42.255907 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2206bf3-6486-4ab1-9e87-8aea1319a409 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:42.340640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:42.367422 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2206bf3-6486-4ab1-9e87-8aea1319a409 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:42.403426 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c24aa8-5b50-4398-a5b5-3e4ab5b8e546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:42.404028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c24aa8-5b50-4398-a5b5-3e4ab5b8e546 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:42.404356 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c24aa8-5b50-4398-a5b5-3e4ab5b8e546 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:42.404624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c24aa8-5b50-4398-a5b5-3e4ab5b8e546 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:42.405291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c24aa8-5b50-4398-a5b5-3e4ab5b8e546 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:42.405592 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c24aa8-5b50-4398-a5b5-3e4ab5b8e546 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:42.407024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c24aa8-5b50-4398-a5b5-3e4ab5b8e546 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:42.416986 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 503/1006] 10.222.0.76 () {56 vars in 1122 bytes} [Wed Aug 24 15:14:42 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:42.418254 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b9898-6aa8-4fb8-ae9a-ef79c8e9a0b2 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:42.433093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:42.527756 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125b9898-6aa8-4fb8-ae9a-ef79c8e9a0b2 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:42.564103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf04a2b-a11a-4ed3-a6c9-b792868e5a4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:42.564710 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf04a2b-a11a-4ed3-a6c9-b792868e5a4f None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:42.564989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf04a2b-a11a-4ed3-a6c9-b792868e5a4f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:42.565259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf04a2b-a11a-4ed3-a6c9-b792868e5a4f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:42.565944 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf04a2b-a11a-4ed3-a6c9-b792868e5a4f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:42.568318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf04a2b-a11a-4ed3-a6c9-b792868e5a4f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:42.584781 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 504/1007] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:14:42 2022] POST /identity/v3/endpoints => generated 325 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:14:43.058846 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343672a-79ec-4688-ae6c-563fd4446a7f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:43.059131 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343672a-79ec-4688-ae6c-563fd4446a7f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:43.059408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343672a-79ec-4688-ae6c-563fd4446a7f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:43.060765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 504/1008] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:43.066824 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871e104-5dc6-4500-ad9c-0acc66ba798f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:43.067215 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871e104-5dc6-4500-ad9c-0acc66ba798f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:43.067561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871e104-5dc6-4500-ad9c-0acc66ba798f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:43.086410 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3871e104-5dc6-4500-ad9c-0acc66ba798f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:43.087957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3871e104-5dc6-4500-ad9c-0acc66ba798f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:43.158810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:43.174264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 505/1009] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:43 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:43.440007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343672a-79ec-4688-ae6c-563fd4446a7f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:43.576022 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343672a-79ec-4688-ae6c-563fd4446a7f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:43.615131 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c93a1-1960-4189-b585-46246dd4a669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:43.615744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c93a1-1960-4189-b585-46246dd4a669 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:43.616022 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c93a1-1960-4189-b585-46246dd4a669 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:43.616335 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c93a1-1960-4189-b585-46246dd4a669 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:43.616950 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c93a1-1960-4189-b585-46246dd4a669 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:43.618417 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6c93a1-1960-4189-b585-46246dd4a669 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:43.630170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:43.642383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 505/1010] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 15:14:43 2022] POST /identity/v3/services => generated 242 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:14:44.160870 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccc696-ab0f-497e-b9aa-1199f0a41dba None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:44.161411 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccc696-ab0f-497e-b9aa-1199f0a41dba None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:44.161411 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccc696-ab0f-497e-b9aa-1199f0a41dba None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:44.162924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 506/1011] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:44.168044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b8f24-f5e8-4718-8e39-a59ed809dc74 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:44.168328 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b8f24-f5e8-4718-8e39-a59ed809dc74 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:44.168597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b8f24-f5e8-4718-8e39-a59ed809dc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:44.188560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f11b8f24-f5e8-4718-8e39-a59ed809dc74 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:44.189845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f11b8f24-f5e8-4718-8e39-a59ed809dc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:44.278663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 506/1012] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:44 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:44.561468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccc696-ab0f-497e-b9aa-1199f0a41dba None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:44.563583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a493fc7-80d4-4012-937c-ff205e793cbb None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:44.564034 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a493fc7-80d4-4012-937c-ff205e793cbb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:44.564136 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a493fc7-80d4-4012-937c-ff205e793cbb None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:44.567010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 507/1013] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:14:44.572556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b23fc3-fb28-4443-8117-2d2dddb5583a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:44.572841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b23fc3-fb28-4443-8117-2d2dddb5583a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:44.573117 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b23fc3-fb28-4443-8117-2d2dddb5583a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:44.594052 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-75b23fc3-fb28-4443-8117-2d2dddb5583a None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:44.595366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-75b23fc3-fb28-4443-8117-2d2dddb5583a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:44.679093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 508/1014] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:44 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:44.686016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccc696-ab0f-497e-b9aa-1199f0a41dba None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:44.726783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8279b-7efb-48fe-8c52-d21675e19f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:44.727422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8279b-7efb-48fe-8c52-d21675e19f3e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:44.727696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8279b-7efb-48fe-8c52-d21675e19f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:44.727952 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f8279b-7efb-48fe-8c52-d21675e19f3e None admin] PATH_INFO: `/v3/services/compute` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:44.728530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8279b-7efb-48fe-8c52-d21675e19f3e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:44.729880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f8279b-7efb-48fe-8c52-d21675e19f3e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-55f8279b-7efb-48fe-8c52-d21675e19f3e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:44.738723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:44.739788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:44.739788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:44.739788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:44.739788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:44.739788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:44.739788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:44.739788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:44.739788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:44.739788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:44.739788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:44.739788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:44.739788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:44.739788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:44.739788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:44.739788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:44.739788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:44.741292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:44.741292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:44.741292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:44.741292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:44.741292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:44.741292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:44.741292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:44.741292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:44.741292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:44.741292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:44.741292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:44.741292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:44.741292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 15:14:44.741292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:44.741292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 507/1015] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:44.745629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b23fc3-fb28-4443-8117-2d2dddb5583a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:44.817565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:44.890402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b23fc3-fb28-4443-8117-2d2dddb5583a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:44.931539 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0589488-d1a8-4d7e-8c6f-a2051667ba43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:44.932490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0589488-d1a8-4d7e-8c6f-a2051667ba43 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:44.932972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0589488-d1a8-4d7e-8c6f-a2051667ba43 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:44.933466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0589488-d1a8-4d7e-8c6f-a2051667ba43 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:44.934509 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0589488-d1a8-4d7e-8c6f-a2051667ba43 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:44.935212 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0589488-d1a8-4d7e-8c6f-a2051667ba43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:44.937106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0589488-d1a8-4d7e-8c6f-a2051667ba43 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:44.945043 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 509/1016] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 15:14:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:44.950925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8279b-7efb-48fe-8c52-d21675e19f3e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:44.967375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0589488-d1a8-4d7e-8c6f-a2051667ba43 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:45.049721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:45.112534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f8279b-7efb-48fe-8c52-d21675e19f3e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:45.120391 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0589488-d1a8-4d7e-8c6f-a2051667ba43 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:45.153726 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:45.163086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e319e-2359-4694-ac71-7a22b1049d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:45.163870 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e319e-2359-4694-ac71-7a22b1049d79 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:45.164407 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e319e-2359-4694-ac71-7a22b1049d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:45.164770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059e319e-2359-4694-ac71-7a22b1049d79 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:45.165628 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e319e-2359-4694-ac71-7a22b1049d79 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:45.166282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e319e-2359-4694-ac71-7a22b1049d79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:45.168481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059e319e-2359-4694-ac71-7a22b1049d79 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:45.176223 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 508/1017] 10.222.0.76 () {56 vars in 1112 bytes} [Wed Aug 24 15:14:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:45.184714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e319e-2359-4694-ac71-7a22b1049d79 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:45.191113 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef9c4f-0159-4520-91fe-85671ecf1572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:45.191776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef9c4f-0159-4520-91fe-85671ecf1572 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:45.192056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef9c4f-0159-4520-91fe-85671ecf1572 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:45.192334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fef9c4f-0159-4520-91fe-85671ecf1572 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:45.193547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef9c4f-0159-4520-91fe-85671ecf1572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:45.196779 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fef9c4f-0159-4520-91fe-85671ecf1572 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:45.202276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 510/1018] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:45.206124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef9c4f-0159-4520-91fe-85671ecf1572 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:45.338664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fef9c4f-0159-4520-91fe-85671ecf1572 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:45.340521 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059e319e-2359-4694-ac71-7a22b1049d79 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:45.376445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044d26af-67b0-4048-baea-d0c98ba78b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:45.376981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d26af-67b0-4048-baea-d0c98ba78b21 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:45.377264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d26af-67b0-4048-baea-d0c98ba78b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:45.377536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044d26af-67b0-4048-baea-d0c98ba78b21 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:45.378186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d26af-67b0-4048-baea-d0c98ba78b21 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:45.380292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044d26af-67b0-4048-baea-d0c98ba78b21 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:45.381047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:45.395925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c197b8-5dd7-4ca5-8daf-df1299735051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:45.396563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c197b8-5dd7-4ca5-8daf-df1299735051 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:45.396850 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c197b8-5dd7-4ca5-8daf-df1299735051 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:45.397127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c197b8-5dd7-4ca5-8daf-df1299735051 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:45.398186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c197b8-5dd7-4ca5-8daf-df1299735051 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:45.399606 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c197b8-5dd7-4ca5-8daf-df1299735051 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:45.420618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 511/1019] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 15:14:45 2022] POST /identity/v3/projects => generated 311 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:14:45.430271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 509/1020] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:14:45 2022] POST /identity/v3/endpoints => generated 334 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:14:45.648090 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3195d7-58f0-456e-b9e1-29b81816edfa None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:45.648583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3195d7-58f0-456e-b9e1-29b81816edfa None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:45.648583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3195d7-58f0-456e-b9e1-29b81816edfa None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:45.649893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 512/1021] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:45.656164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab42d2-d479-4f0c-ab20-6c5e2eaff00f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:45.656426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab42d2-d479-4f0c-ab20-6c5e2eaff00f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:45.656739 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab42d2-d479-4f0c-ab20-6c5e2eaff00f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:45.676306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3dab42d2-d479-4f0c-ab20-6c5e2eaff00f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:45.677499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3dab42d2-d479-4f0c-ab20-6c5e2eaff00f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:45.770495 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 510/1022] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:45 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:46.040482 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3195d7-58f0-456e-b9e1-29b81816edfa None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:46.096008 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:46.179969 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3195d7-58f0-456e-b9e1-29b81816edfa None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:46.218808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b709b3c-7b66-4b0a-8f7d-151dca5fcd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:46.219438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b709b3c-7b66-4b0a-8f7d-151dca5fcd9e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:46.219717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b709b3c-7b66-4b0a-8f7d-151dca5fcd9e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:46.219988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b709b3c-7b66-4b0a-8f7d-151dca5fcd9e None admin] PATH_INFO: `/v3/services/image` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:46.220567 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b709b3c-7b66-4b0a-8f7d-151dca5fcd9e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:46.221931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b709b3c-7b66-4b0a-8f7d-151dca5fcd9e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-8b709b3c-7b66-4b0a-8f7d-151dca5fcd9e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:46.233965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:46.235049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:46.235049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:46.235049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:46.235049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:46.235049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:46.235049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:46.235049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:46.235049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:46.235049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:46.235049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:46.235049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:46.235049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:46.235049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:46.235049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:46.235049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:46.235049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:46.236627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:46.236627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:46.236627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:46.236627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:46.236627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:46.236627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:46.236627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:46.236627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:46.236627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:46.236627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:46.236627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:46.236627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:46.236627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:14:46.236627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:46.244283 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 513/1023] 10.222.0.76 () {56 vars in 1105 bytes} [Wed Aug 24 15:14:46 2022] GET /identity/v3/services/image => generated 86 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:46.244444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab42d2-d479-4f0c-ab20-6c5e2eaff00f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:46.358018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab42d2-d479-4f0c-ab20-6c5e2eaff00f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:46.367466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:46.403212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9285f-b0dc-44e6-b61b-bef9b7655fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:46.404083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9285f-b0dc-44e6-b61b-bef9b7655fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:46.404445 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9285f-b0dc-44e6-b61b-bef9b7655fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:46.404935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e9285f-b0dc-44e6-b61b-bef9b7655fe7 None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:46.405563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9285f-b0dc-44e6-b61b-bef9b7655fe7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:46.406213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9285f-b0dc-44e6-b61b-bef9b7655fe7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:46.407665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e9285f-b0dc-44e6-b61b-bef9b7655fe7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:46.417054 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 511/1024] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:46.418580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b709b3c-7b66-4b0a-8f7d-151dca5fcd9e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:46.540493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b709b3c-7b66-4b0a-8f7d-151dca5fcd9e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:46.542272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:46.579650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64df2ad-1565-4fe1-903a-260dc56e37b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:46.580282 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64df2ad-1565-4fe1-903a-260dc56e37b3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:46.580556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64df2ad-1565-4fe1-903a-260dc56e37b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:46.580826 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64df2ad-1565-4fe1-903a-260dc56e37b3 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:46.581628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64df2ad-1565-4fe1-903a-260dc56e37b3 None admin] RBAC: Adding query filter params (type=image) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:46.581934 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64df2ad-1565-4fe1-903a-260dc56e37b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:46.583399 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64df2ad-1565-4fe1-903a-260dc56e37b3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:46.589413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 514/1025] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:46.595568 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9285f-b0dc-44e6-b61b-bef9b7655fe7 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:46.700628 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e9285f-b0dc-44e6-b61b-bef9b7655fe7 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:46.737136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef62588d-3899-4950-ac35-b5b794a5bffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:46.738303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62588d-3899-4950-ac35-b5b794a5bffe None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:46.738590 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62588d-3899-4950-ac35-b5b794a5bffe None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:46.738862 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef62588d-3899-4950-ac35-b5b794a5bffe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:46.739561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62588d-3899-4950-ac35-b5b794a5bffe None admin] RBAC: Adding query filter params (interface=public, service_id=0ba59cb1a24140a79e1867e768b24b60, region_id=RegionOne) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:46.739870 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62588d-3899-4950-ac35-b5b794a5bffe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:46.741237 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef62588d-3899-4950-ac35-b5b794a5bffe None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:46.746456 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 512/1026] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 24 15:14:46 2022] GET /identity/v3/endpoints?service_id=0ba59cb1a24140a79e1867e768b24b60&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:46.752011 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64df2ad-1565-4fe1-903a-260dc56e37b3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:46.855636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64df2ad-1565-4fe1-903a-260dc56e37b3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:46.891356 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58eabe-b2a4-4e94-a9db-6fc41de4f8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:46.891966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58eabe-b2a4-4e94-a9db-6fc41de4f8a6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:46.892243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58eabe-b2a4-4e94-a9db-6fc41de4f8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:46.892511 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58eabe-b2a4-4e94-a9db-6fc41de4f8a6 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:46.893105 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58eabe-b2a4-4e94-a9db-6fc41de4f8a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:46.894608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58eabe-b2a4-4e94-a9db-6fc41de4f8a6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:46.910209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 515/1027] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 15:14:46 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:14:47.470178 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3d70d-b5cf-4783-a91d-032b225f80e1 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:47.470759 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3d70d-b5cf-4783-a91d-032b225f80e1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:47.470759 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3d70d-b5cf-4783-a91d-032b225f80e1 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:47.472105 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 513/1028] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:47.478146 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392032cb-15b6-4a54-b640-f17337f4c64f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:47.478457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392032cb-15b6-4a54-b640-f17337f4c64f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:47.478730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392032cb-15b6-4a54-b640-f17337f4c64f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:47.500116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-392032cb-15b6-4a54-b640-f17337f4c64f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:47.500116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-392032cb-15b6-4a54-b640-f17337f4c64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:47.525657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:47.533794 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a9093-fd17-466e-8b4e-93a0a977ef2f None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:47.534110 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a9093-fd17-466e-8b4e-93a0a977ef2f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:47.534382 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438a9093-fd17-466e-8b4e-93a0a977ef2f None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:47.536552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 514/1029] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:47.541644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cc496-a9d1-4808-bd3e-15b03ee52866 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:47.541906 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cc496-a9d1-4808-bd3e-15b03ee52866 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:47.542181 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cc496-a9d1-4808-bd3e-15b03ee52866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:47.559686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-1d8cc496-a9d1-4808-bd3e-15b03ee52866 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:47.560873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1d8cc496-a9d1-4808-bd3e-15b03ee52866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:47.580148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 516/1030] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:47.589614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:47.634184 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 515/1031] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:47 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:47.848899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392032cb-15b6-4a54-b640-f17337f4c64f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:47.904307 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cc496-a9d1-4808-bd3e-15b03ee52866 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:47.963793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392032cb-15b6-4a54-b640-f17337f4c64f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:48.010738 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39537788-6dc2-45e5-a591-05b9c0b3789e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:48.011751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39537788-6dc2-45e5-a591-05b9c0b3789e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:48.012200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39537788-6dc2-45e5-a591-05b9c0b3789e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:48.012638 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39537788-6dc2-45e5-a591-05b9c0b3789e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:48.013751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39537788-6dc2-45e5-a591-05b9c0b3789e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:48.015204 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39537788-6dc2-45e5-a591-05b9c0b3789e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:48.016524 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cc496-a9d1-4808-bd3e-15b03ee52866 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:48.017107 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 517/1032] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:48.023288 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39537788-6dc2-45e5-a591-05b9c0b3789e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:48.056595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1aacf-70a9-4855-baeb-65cc1a0231df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:48.057228 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1aacf-70a9-4855-baeb-65cc1a0231df None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:48.057502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1aacf-70a9-4855-baeb-65cc1a0231df None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:48.057774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1aacf-70a9-4855-baeb-65cc1a0231df None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:48.058077 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:48.068742 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa1aacf-70a9-4855-baeb-65cc1a0231df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:48.073490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-5aa1aacf-70a9-4855-baeb-65cc1a0231df None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:14:48.073490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:48.073490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:48.073490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:48.073490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:48.073490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:48.073490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:48.073490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:48.073490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:48.073490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:48.073490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:48.073490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:48.073490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:48.073490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:48.073490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:48.073490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:48.074580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:48.074580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:48.074580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:48.074580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:48.074580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:48.074580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:48.074580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:48.074580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:48.074580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:48.074580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:48.074580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:48.074580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:48.074580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:48.074580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:48.074580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:48.074580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:48.075413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:48.075413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:48.075413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:48.075413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:48.075413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:48.075413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:48.075413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:48.075413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:48.075413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:48.075413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:48.075413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:14:48.075413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:48.079675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 516/1033] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 24 15:14:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:48.080582 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1aacf-70a9-4855-baeb-65cc1a0231df None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:48.115153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:48.139759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39537788-6dc2-45e5-a591-05b9c0b3789e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:48.186321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390c4c5-4fe5-4c73-8aae-89a42e3a9f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:48.187300 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390c4c5-4fe5-4c73-8aae-89a42e3a9f29 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:48.187805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390c4c5-4fe5-4c73-8aae-89a42e3a9f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:48.188270 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f390c4c5-4fe5-4c73-8aae-89a42e3a9f29 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:48.189156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390c4c5-4fe5-4c73-8aae-89a42e3a9f29 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:48.190698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f390c4c5-4fe5-4c73-8aae-89a42e3a9f29 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:48.213223 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1aacf-70a9-4855-baeb-65cc1a0231df None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:48.224554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 518/1034] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:14:48 2022] POST /identity/v3/users => generated 285 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:14:48.270500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b72423-7e95-4eaa-87fd-ee379c2f2e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:48.271132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b72423-7e95-4eaa-87fd-ee379c2f2e5b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:48.271413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b72423-7e95-4eaa-87fd-ee379c2f2e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:48.271683 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b72423-7e95-4eaa-87fd-ee379c2f2e5b None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:48.272383 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b72423-7e95-4eaa-87fd-ee379c2f2e5b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:48.272684 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b72423-7e95-4eaa-87fd-ee379c2f2e5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:48.274313 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b72423-7e95-4eaa-87fd-ee379c2f2e5b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:48.290470 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 517/1035] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 24 15:14:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:48.291904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390c4c5-4fe5-4c73-8aae-89a42e3a9f29 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:48.410239 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:48.433938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f390c4c5-4fe5-4c73-8aae-89a42e3a9f29 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:48.471606 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b15b8c-38d4-4d94-8f7e-1464ab1b5832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:48.471606 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b15b8c-38d4-4d94-8f7e-1464ab1b5832 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:48.471606 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b15b8c-38d4-4d94-8f7e-1464ab1b5832 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:48.471606 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b15b8c-38d4-4d94-8f7e-1464ab1b5832 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:48.484038 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b15b8c-38d4-4d94-8f7e-1464ab1b5832 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:48.485697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b15b8c-38d4-4d94-8f7e-1464ab1b5832 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:48.509759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-30b15b8c-38d4-4d94-8f7e-1464ab1b5832 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:48.509759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:48.509759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:48.509759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:48.509759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:48.509759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:48.509759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:48.509759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:48.509759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:48.509759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:48.509759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:48.509759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:48.509759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:48.509759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:48.509759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:48.509759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:48.511479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:48.513127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:48.513127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:48.513127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:48.513127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 519/1036] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:48.520367 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b72423-7e95-4eaa-87fd-ee379c2f2e5b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:48.644828 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:48.664027 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b72423-7e95-4eaa-87fd-ee379c2f2e5b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:48.710987 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348cdf1-9a81-4c62-b4cf-7e0710a2d07d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:48.712068 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348cdf1-9a81-4c62-b4cf-7e0710a2d07d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:48.712534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348cdf1-9a81-4c62-b4cf-7e0710a2d07d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:48.713020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348cdf1-9a81-4c62-b4cf-7e0710a2d07d None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:48.713863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7348cdf1-9a81-4c62-b4cf-7e0710a2d07d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:48.714404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7348cdf1-9a81-4c62-b4cf-7e0710a2d07d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:48.716014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7348cdf1-9a81-4c62-b4cf-7e0710a2d07d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:48.732352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 518/1037] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:48.739583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b15b8c-38d4-4d94-8f7e-1464ab1b5832 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:48.879207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b15b8c-38d4-4d94-8f7e-1464ab1b5832 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:48.909246 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531419ed-182d-4c0a-a53d-05ae7aef9468 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:48.909802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531419ed-182d-4c0a-a53d-05ae7aef9468 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:48.910324 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531419ed-182d-4c0a-a53d-05ae7aef9468 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:48.912728 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 519/1038] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:48.921175 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1c902-609c-4ab2-9abf-68cf9648220a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:48.921639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1c902-609c-4ab2-9abf-68cf9648220a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:48.922095 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1c902-609c-4ab2-9abf-68cf9648220a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:48.939359 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ec897-7da8-42cd-8488-257fc17e80db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:48.940338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ec897-7da8-42cd-8488-257fc17e80db None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:48.941441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ec897-7da8-42cd-8488-257fc17e80db None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:48.942474 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21ec897-7da8-42cd-8488-257fc17e80db None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:48.947012 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:48.949449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d8f1c902-609c-4ab2-9abf-68cf9648220a None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:48.951481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d8f1c902-609c-4ab2-9abf-68cf9648220a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:48.963639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ec897-7da8-42cd-8488-257fc17e80db None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:48.966604 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21ec897-7da8-42cd-8488-257fc17e80db None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:48.983706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d21ec897-7da8-42cd-8488-257fc17e80db None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:14:48.983706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:48.983706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:48.983706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:48.983706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:48.983706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:48.983706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:48.983706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:48.983706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:48.983706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:48.983706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:48.983706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:48.983706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:48.983706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:48.983706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:48.983706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:48.985593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:48.988994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:48.988994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:48.988994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:48.988994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:48.988994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:48.988994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:48.988994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:48.988994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:48.988994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:14:48.988994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:48.988994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 520/1039] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 15:14:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:48.994092 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ec897-7da8-42cd-8488-257fc17e80db None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:49.051363 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 520/1040] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:48 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:49.124185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21ec897-7da8-42cd-8488-257fc17e80db None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:49.170653 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d032d-9d83-4441-8730-2519a37850d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:49.171704 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d032d-9d83-4441-8730-2519a37850d8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:49.172087 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d032d-9d83-4441-8730-2519a37850d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:49.172594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d032d-9d83-4441-8730-2519a37850d8 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:49.173861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d032d-9d83-4441-8730-2519a37850d8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:49.174454 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d032d-9d83-4441-8730-2519a37850d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:49.176925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2d032d-9d83-4441-8730-2519a37850d8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:49.201406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 521/1041] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 15:14:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:49.209888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1c902-609c-4ab2-9abf-68cf9648220a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:49.350342 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:49.355419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d032d-9d83-4441-8730-2519a37850d8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:49.394297 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1c902-609c-4ab2-9abf-68cf9648220a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:49.432454 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:49.452715 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9dbaba-b93f-41fa-ae78-7bc395704e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:49.453826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9dbaba-b93f-41fa-ae78-7bc395704e6a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:49.454492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9dbaba-b93f-41fa-ae78-7bc395704e6a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:49.454827 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9dbaba-b93f-41fa-ae78-7bc395704e6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:49.503723 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d032d-9d83-4441-8730-2519a37850d8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:49.558619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9146-2b91-4cc3-bb2c-ebcc77260140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:49.559790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9146-2b91-4cc3-bb2c-ebcc77260140 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:49.560148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9146-2b91-4cc3-bb2c-ebcc77260140 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:49.560553 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce9146-2b91-4cc3-bb2c-ebcc77260140 None admin] PATH_INFO: `/v3/services/image` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:49.561424 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce9146-2b91-4cc3-bb2c-ebcc77260140 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:49.563436 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce9146-2b91-4cc3-bb2c-ebcc77260140 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:49.575270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9dbaba-b93f-41fa-ae78-7bc395704e6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-f7ce9146-2b91-4cc3-bb2c-ebcc77260140 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:49.575832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:49.576913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:49.576913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:49.576913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:49.576913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:49.576913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:49.576913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:49.576913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:14:49.576913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:14:49.576913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:14:49.576913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:14:49.576913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:49.576913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:49.576913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:49.576913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:49.576913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:49.576913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:49.577980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:49.577980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:49.577980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:49.577980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:49.577980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:49.577980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:49.577980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:49.577980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:49.577980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:49.577980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:49.577980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:14:49.577980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:14:49.577980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 15:14:49.577980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:49.577980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 522/1042] 10.222.0.76 () {56 vars in 1105 bytes} [Wed Aug 24 15:14:49 2022] GET /identity/v3/services/image => generated 86 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:49.580553 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9dbaba-b93f-41fa-ae78-7bc395704e6a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:49.583298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9146-2b91-4cc3-bb2c-ebcc77260140 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:49.684744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:49.720876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 521/1043] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:14:49 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:14:49.728336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9dbaba-b93f-41fa-ae78-7bc395704e6a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:49.729695 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce9146-2b91-4cc3-bb2c-ebcc77260140 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:49.751028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:49.776868 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94135376-d44c-42e1-8759-52b4a2a3c966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:49.777776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94135376-d44c-42e1-8759-52b4a2a3c966 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:49.778204 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94135376-d44c-42e1-8759-52b4a2a3c966 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:49.778656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94135376-d44c-42e1-8759-52b4a2a3c966 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:49.779644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94135376-d44c-42e1-8759-52b4a2a3c966 None admin] RBAC: Adding query filter params (name=image) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:49.780113 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94135376-d44c-42e1-8759-52b4a2a3c966 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:49.782384 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94135376-d44c-42e1-8759-52b4a2a3c966 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:49.788178 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 523/1044] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:49.794295 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94135376-d44c-42e1-8759-52b4a2a3c966 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:49.891776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9dbaba-b93f-41fa-ae78-7bc395704e6a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:49.939169 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fe8a1-e8db-43eb-8037-aa934acd9ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:49.940143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fe8a1-e8db-43eb-8037-aa934acd9ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:49.940647 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fe8a1-e8db-43eb-8037-aa934acd9ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:49.941663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fe8a1-e8db-43eb-8037-aa934acd9ea9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:49.949060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94135376-d44c-42e1-8759-52b4a2a3c966 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:49.953658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fe8a1-e8db-43eb-8037-aa934acd9ea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:49.955134 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fe8a1-e8db-43eb-8037-aa934acd9ea9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:49.970942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-7e1fe8a1-e8db-43eb-8037-aa934acd9ea9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:49.970942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:49.970942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:49.970942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:49.970942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:49.970942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:49.970942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:49.970942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:49.970942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:49.970942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:49.970942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:49.970942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:49.970942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:49.970942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:49.970942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:49.970942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:49.972532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:49.973428 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:49.973428 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:49.973428 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:49.977419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 522/1045] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:49.978282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fe8a1-e8db-43eb-8037-aa934acd9ea9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:49.989716 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efab9a-7550-4c85-84e7-b46f99736389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:49.990384 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efab9a-7550-4c85-84e7-b46f99736389 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:49.990669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efab9a-7550-4c85-84e7-b46f99736389 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:49.990945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14efab9a-7550-4c85-84e7-b46f99736389 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:49.991578 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efab9a-7550-4c85-84e7-b46f99736389 None admin] RBAC: Adding query filter params (type=image) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:49.991890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efab9a-7550-4c85-84e7-b46f99736389 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:49.993272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14efab9a-7550-4c85-84e7-b46f99736389 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:49.998282 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 524/1046] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:50.004001 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efab9a-7550-4c85-84e7-b46f99736389 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:50.017123 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:50.055319 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:50.098242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fe8a1-e8db-43eb-8037-aa934acd9ea9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:50.120516 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14efab9a-7550-4c85-84e7-b46f99736389 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:50.137259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ec709-703d-47ed-a729-74bd6753cd06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:50.137871 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ec709-703d-47ed-a729-74bd6753cd06 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:50.138148 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ec709-703d-47ed-a729-74bd6753cd06 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:50.138450 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4ec709-703d-47ed-a729-74bd6753cd06 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:50.139089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ec709-703d-47ed-a729-74bd6753cd06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:50.139385 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ec709-703d-47ed-a729-74bd6753cd06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:50.140748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4ec709-703d-47ed-a729-74bd6753cd06 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:50.158467 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 523/1047] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:50.159397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ec709-703d-47ed-a729-74bd6753cd06 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:50.163694 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c001080-494e-4a61-9d61-078637e48417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:50.164304 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c001080-494e-4a61-9d61-078637e48417 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:50.164582 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c001080-494e-4a61-9d61-078637e48417 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:50.164853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c001080-494e-4a61-9d61-078637e48417 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:50.165610 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c001080-494e-4a61-9d61-078637e48417 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:50.167061 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c001080-494e-4a61-9d61-078637e48417 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:50.191725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 525/1048] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:14:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:14:50.274666 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:50.291617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4ec709-703d-47ed-a729-74bd6753cd06 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:50.337711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2013347-ddc7-4967-824a-f3955b82c428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:50.339330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2013347-ddc7-4967-824a-f3955b82c428 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:50.339330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2013347-ddc7-4967-824a-f3955b82c428 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:50.339330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2013347-ddc7-4967-824a-f3955b82c428 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:50.346542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2013347-ddc7-4967-824a-f3955b82c428 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:50.348598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2013347-ddc7-4967-824a-f3955b82c428 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:50.360754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-b2013347-ddc7-4967-824a-f3955b82c428 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:50.360754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:50.360754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:50.360754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:50.360754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:50.360754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:50.360754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:50.360754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:50.360754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:50.360754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:50.360754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:50.360754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:50.360754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:50.360754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:50.360754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:50.361657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:50.362677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:50.362677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:50.362677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:50.362677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:50.362677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:50.362677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:50.362677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:50.362677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:50.362677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 524/1049] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:50.367894 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c001080-494e-4a61-9d61-078637e48417 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:50.473077 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c001080-494e-4a61-9d61-078637e48417 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:50.474576 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:50.513231 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae15168-2644-431c-bd5f-e33994ba053d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:50.513853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae15168-2644-431c-bd5f-e33994ba053d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:50.514133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae15168-2644-431c-bd5f-e33994ba053d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:50.514437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae15168-2644-431c-bd5f-e33994ba053d None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:50.515159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae15168-2644-431c-bd5f-e33994ba053d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:50.515467 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae15168-2644-431c-bd5f-e33994ba053d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:50.517003 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae15168-2644-431c-bd5f-e33994ba053d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:50.534266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 526/1050] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:50.537779 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2013347-ddc7-4967-824a-f3955b82c428 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:50.641643 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2013347-ddc7-4967-824a-f3955b82c428 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:50.677294 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175c2ba-da6b-4f97-9650-4352bf07b829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:50.677903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175c2ba-da6b-4f97-9650-4352bf07b829 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:50.678174 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175c2ba-da6b-4f97-9650-4352bf07b829 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:50.678470 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175c2ba-da6b-4f97-9650-4352bf07b829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:50.679186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175c2ba-da6b-4f97-9650-4352bf07b829 None admin] RBAC: Adding query filter params (role.id=c87fe7f37cb246e8af3bc71057ec5048, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=8a5fd31a2db944f596912e4f0aa26835) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:50.679486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175c2ba-da6b-4f97-9650-4352bf07b829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:50.680891 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4175c2ba-da6b-4f97-9650-4352bf07b829 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:50.694266 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 525/1051] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:14:50 2022] GET /identity/v3/role_assignments?user.id=8a5fd31a2db944f596912e4f0aa26835&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=c87fe7f37cb246e8af3bc71057ec5048 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:50.824462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb908955-d0e2-4624-8fa0-d87aedbd1512 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:50.824883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb908955-d0e2-4624-8fa0-d87aedbd1512 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:50.825356 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb908955-d0e2-4624-8fa0-d87aedbd1512 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:50.827947 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 527/1052] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:50.836500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe73995-401b-47b6-b7d0-137dc4e238af None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:50.836779 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe73995-401b-47b6-b7d0-137dc4e238af None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:50.837049 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe73995-401b-47b6-b7d0-137dc4e238af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:50.856413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7fe73995-401b-47b6-b7d0-137dc4e238af None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:50.857879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7fe73995-401b-47b6-b7d0-137dc4e238af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:50.860710 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:50.934249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 526/1053] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:51.200889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb908955-d0e2-4624-8fa0-d87aedbd1512 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:51.308348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb908955-d0e2-4624-8fa0-d87aedbd1512 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:51.344184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea7bd5-3e94-4931-a89f-25fb338012f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:51.344815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea7bd5-3e94-4931-a89f-25fb338012f0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:51.348792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea7bd5-3e94-4931-a89f-25fb338012f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:51.348792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea7bd5-3e94-4931-a89f-25fb338012f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:51.359757 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea7bd5-3e94-4931-a89f-25fb338012f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:51.365985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2bea7bd5-3e94-4931-a89f-25fb338012f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:51.365985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:51.365985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:51.365985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:51.365985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:51.365985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:51.365985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:51.365985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:51.365985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:51.365985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:51.365985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:51.365985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:51.365985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:51.365985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:51.365985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:51.365985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:51.367017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:51.367017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:51.367017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:51.367017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:51.367017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:51.367017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:51.367017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:51.367017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:51.367017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:51.367017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:51.367017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:51.367017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:51.367017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:51.367017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:51.367017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:51.367017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:51.367863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:51.367863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:51.367863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:51.367863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:51.367863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:51.367863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:51.367863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:51.367863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:51.367863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:51.367863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:51.367863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:51.367863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:51.370588 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 528/1054] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:14:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:51.374515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe73995-401b-47b6-b7d0-137dc4e238af None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:51.476132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe73995-401b-47b6-b7d0-137dc4e238af None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:51.517947 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e4f58-6a52-47a0-b18a-a9ee6646c4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:51.517947 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e4f58-6a52-47a0-b18a-a9ee6646c4f8 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:51.517947 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e4f58-6a52-47a0-b18a-a9ee6646c4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:51.518372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e4f58-6a52-47a0-b18a-a9ee6646c4f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:51.519487 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e4f58-6a52-47a0-b18a-a9ee6646c4f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:51.519737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e4f58-6a52-47a0-b18a-a9ee6646c4f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:51.521247 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e4f58-6a52-47a0-b18a-a9ee6646c4f8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:51.529762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:51.535145 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 527/1055] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:14:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:51.541007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea7bd5-3e94-4931-a89f-25fb338012f0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:51.568785 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:51.655269 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea7bd5-3e94-4931-a89f-25fb338012f0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:51.691957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2087b-a792-41a3-b982-864150385042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:51.692594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2087b-a792-41a3-b982-864150385042 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:51.692871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2087b-a792-41a3-b982-864150385042 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:51.693141 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee2087b-a792-41a3-b982-864150385042 None admin] PATH_INFO: `/v3/users/2f637462c68c4714bd92c97f7f05f72d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:51.707638 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2087b-a792-41a3-b982-864150385042 None admin] RBAC: Authorizing `identity:get_user(user_id=2f637462c68c4714bd92c97f7f05f72d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:51.709082 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee2087b-a792-41a3-b982-864150385042 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:51.710981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 529/1056] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:14:51 2022] GET /identity/v3/users/2f637462c68c4714bd92c97f7f05f72d => generated 285 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:51.716613 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e4f58-6a52-47a0-b18a-a9ee6646c4f8 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:51.823021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e4f58-6a52-47a0-b18a-a9ee6646c4f8 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:51.857798 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f1e28-3cbc-42c1-9b56-02c2760fa1d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:51.858489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f1e28-3cbc-42c1-9b56-02c2760fa1d4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:51.858777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f1e28-3cbc-42c1-9b56-02c2760fa1d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:51.859048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f1e28-3cbc-42c1-9b56-02c2760fa1d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:51.932126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:51.943185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f1e28-3cbc-42c1-9b56-02c2760fa1d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:51.944720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f1e28-3cbc-42c1-9b56-02c2760fa1d4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:52.076496 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 528/1057] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:14:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:14:52.077872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2087b-a792-41a3-b982-864150385042 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:52.156046 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfe7a9-9772-4e12-9167-70474e92f938 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:52.156327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfe7a9-9772-4e12-9167-70474e92f938 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:52.156594 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfe7a9-9772-4e12-9167-70474e92f938 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:52.157942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 529/1058] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:52.163325 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f396b-97bc-4227-a4b8-1b971d126659 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:52.163606 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f396b-97bc-4227-a4b8-1b971d126659 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:52.163876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5f396b-97bc-4227-a4b8-1b971d126659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:52.182885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6b5f396b-97bc-4227-a4b8-1b971d126659 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:52.184214 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6b5f396b-97bc-4227-a4b8-1b971d126659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:52.195074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:52.217752 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee2087b-a792-41a3-b982-864150385042 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:52.258993 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7072dd2-2833-4236-b4d3-2ebb3bd691fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:52.259883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7072dd2-2833-4236-b4d3-2ebb3bd691fd None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:52.260332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7072dd2-2833-4236-b4d3-2ebb3bd691fd None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:52.260769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7072dd2-2833-4236-b4d3-2ebb3bd691fd None admin] PATH_INFO: `/v3/projects/c3e0ded9bb254e2888b04c1f0d31898d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:52.262026 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7072dd2-2833-4236-b4d3-2ebb3bd691fd None admin] RBAC: Authorizing `identity:get_project(project_id=c3e0ded9bb254e2888b04c1f0d31898d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:52.263503 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7072dd2-2833-4236-b4d3-2ebb3bd691fd None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:52.272567 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 530/1059] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:14:52 2022] GET /identity/v3/projects/c3e0ded9bb254e2888b04c1f0d31898d => generated 311 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:52.272803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 530/1060] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:52.273646 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7072dd2-2833-4236-b4d3-2ebb3bd691fd None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:52.397940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7072dd2-2833-4236-b4d3-2ebb3bd691fd None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:52.439801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd4ee2d-4b32-462f-b015-0e6a474ef532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:52.440416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4ee2d-4b32-462f-b015-0e6a474ef532 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:52.440724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4ee2d-4b32-462f-b015-0e6a474ef532 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:52.440997 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd4ee2d-4b32-462f-b015-0e6a474ef532 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:52.441731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd4ee2d-4b32-462f-b015-0e6a474ef532 None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=c3e0ded9bb254e2888b04c1f0d31898d, user.id=2f637462c68c4714bd92c97f7f05f72d) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:52.442036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd4ee2d-4b32-462f-b015-0e6a474ef532 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:52.443503 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd4ee2d-4b32-462f-b015-0e6a474ef532 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:52.458264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 531/1061] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:14:52 2022] GET /identity/v3/role_assignments?user.id=2f637462c68c4714bd92c97f7f05f72d&scope.project.id=c3e0ded9bb254e2888b04c1f0d31898d&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:52.543586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f396b-97bc-4227-a4b8-1b971d126659 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:52.583574 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:52.603200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943cf2be-ab54-4c12-927f-3eb171d5b15b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:52.603482 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943cf2be-ab54-4c12-927f-3eb171d5b15b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:52.603754 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943cf2be-ab54-4c12-927f-3eb171d5b15b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:52.605108 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 532/1062] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:14:52.610943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ff6db-f960-4732-b535-0ef4aff77de7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:52.611207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ff6db-f960-4732-b535-0ef4aff77de7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:52.611485 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712ff6db-f960-4732-b535-0ef4aff77de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:52.632973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-712ff6db-f960-4732-b535-0ef4aff77de7 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:52.634169 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-712ff6db-f960-4732-b535-0ef4aff77de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:52.648350 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:52.672524 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5f396b-97bc-4227-a4b8-1b971d126659 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:52.719738 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308831c3-9eba-430e-8435-4e120aae363b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:52.720596 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308831c3-9eba-430e-8435-4e120aae363b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:52.720916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308831c3-9eba-430e-8435-4e120aae363b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:52.721189 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308831c3-9eba-430e-8435-4e120aae363b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:52.722701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 533/1063] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:14:52.804181 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308831c3-9eba-430e-8435-4e120aae363b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:52.805643 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308831c3-9eba-430e-8435-4e120aae363b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:52.846295 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:52.950830 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 531/1064] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:14:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:14:52.956766 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ff6db-f960-4732-b535-0ef4aff77de7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:52.992065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308831c3-9eba-430e-8435-4e120aae363b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:53.090522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712ff6db-f960-4732-b535-0ef4aff77de7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:53.115207 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308831c3-9eba-430e-8435-4e120aae363b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:53.135604 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6932e6-3f17-45d7-bfd9-dc33b3589a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:53.136234 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6932e6-3f17-45d7-bfd9-dc33b3589a03 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:53.136514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6932e6-3f17-45d7-bfd9-dc33b3589a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:53.136783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6932e6-3f17-45d7-bfd9-dc33b3589a03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:53.149445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6932e6-3f17-45d7-bfd9-dc33b3589a03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:53.150462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:53.150937 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6932e6-3f17-45d7-bfd9-dc33b3589a03 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:53.153313 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:53.160108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f7c7b-516a-4c0e-bb92-3a378f539f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:53.160995 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f7c7b-516a-4c0e-bb92-3a378f539f30 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:53.161740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f7c7b-516a-4c0e-bb92-3a378f539f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:53.162207 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8f7c7b-516a-4c0e-bb92-3a378f539f30 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:53.174397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-2d6932e6-3f17-45d7-bfd9-dc33b3589a03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:53.174397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:53.174397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:53.174397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:53.174397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:53.174397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:53.174397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:53.174397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:53.174397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:53.174397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:53.174397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:53.174397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:53.174397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:53.174397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:53.174397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:53.174397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:53.175456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:53.176342 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:53.176342 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:53.176342 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:53.176342 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 534/1065] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:53.181727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6932e6-3f17-45d7-bfd9-dc33b3589a03 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:53.183368 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8f7c7b-516a-4c0e-bb92-3a378f539f30 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:53.192571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-1a8f7c7b-516a-4c0e-bb92-3a378f539f30 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:14:53.192571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:53.192571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:53.192571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:53.192571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:53.192571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:53.192571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:53.192571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:53.192571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:53.192571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:53.192571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:53.192571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:53.192571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:53.192571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:53.192571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:53.192571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:53.194570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:53.194570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:53.194570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:53.194570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:53.194570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:53.194570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:53.194570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:53.194570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:53.194570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:53.194570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:53.194570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:53.194570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:53.194570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:53.194570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:53.194570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:53.194570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:53.195714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:53.195714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:53.195714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:53.195714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:53.195714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:53.195714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:53.195714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:53.195714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:53.195714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:53.195714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:53.195714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:14:53.195714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:53.195714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 532/1066] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 24 15:14:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:53.204671 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f7c7b-516a-4c0e-bb92-3a378f539f30 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:53.347206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6932e6-3f17-45d7-bfd9-dc33b3589a03 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:53.366960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8f7c7b-516a-4c0e-bb92-3a378f539f30 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:53.396651 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b4ea9-297e-4771-935b-b648fb3288c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:53.397534 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b4ea9-297e-4771-935b-b648fb3288c5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:53.397804 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b4ea9-297e-4771-935b-b648fb3288c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:53.398070 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b4ea9-297e-4771-935b-b648fb3288c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:53.398762 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b4ea9-297e-4771-935b-b648fb3288c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:53.399063 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b4ea9-297e-4771-935b-b648fb3288c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:53.400422 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b4ea9-297e-4771-935b-b648fb3288c5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:53.412165 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70dc91-6c6b-4b40-abef-f66655a5ebaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:53.412879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70dc91-6c6b-4b40-abef-f66655a5ebaa None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:53.413234 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70dc91-6c6b-4b40-abef-f66655a5ebaa None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:53.413573 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb70dc91-6c6b-4b40-abef-f66655a5ebaa None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:53.414353 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb70dc91-6c6b-4b40-abef-f66655a5ebaa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:53.414661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb70dc91-6c6b-4b40-abef-f66655a5ebaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:53.416061 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb70dc91-6c6b-4b40-abef-f66655a5ebaa None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:53.416752 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 535/1067] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:53.430766 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 533/1068] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 24 15:14:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:53.436739 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b4ea9-297e-4771-935b-b648fb3288c5 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:53.512616 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:53.576051 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b4ea9-297e-4771-935b-b648fb3288c5 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:53.612502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3486d9f-f736-45b6-8e8b-96b2972b0bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:53.613460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3486d9f-f736-45b6-8e8b-96b2972b0bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:53.613743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3486d9f-f736-45b6-8e8b-96b2972b0bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:53.614011 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3486d9f-f736-45b6-8e8b-96b2972b0bc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:53.625120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3486d9f-f736-45b6-8e8b-96b2972b0bc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:53.626754 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3486d9f-f736-45b6-8e8b-96b2972b0bc1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:53.642271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-c3486d9f-f736-45b6-8e8b-96b2972b0bc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:53.642271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:53.642271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:53.642271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:53.642271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:53.642271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:53.642271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:53.642271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:53.642271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:53.642271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:53.642271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:53.642271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:53.642271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:53.642271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:53.642271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:53.642271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:53.643460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:53.644621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:53.644621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:53.644621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:53.644621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 536/1069] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:53.650123 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70dc91-6c6b-4b40-abef-f66655a5ebaa None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:53.720439 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:53.780239 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb70dc91-6c6b-4b40-abef-f66655a5ebaa None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:53.815910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69b16e-43b2-4677-a560-94cba3361c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:53.816642 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69b16e-43b2-4677-a560-94cba3361c7a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:53.816929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69b16e-43b2-4677-a560-94cba3361c7a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:53.817266 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de69b16e-43b2-4677-a560-94cba3361c7a None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:53.817905 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69b16e-43b2-4677-a560-94cba3361c7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:53.818202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69b16e-43b2-4677-a560-94cba3361c7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:53.819610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de69b16e-43b2-4677-a560-94cba3361c7a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:53.832732 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 534/1070] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:53.838585 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3486d9f-f736-45b6-8e8b-96b2972b0bc1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:53.967831 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3486d9f-f736-45b6-8e8b-96b2972b0bc1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:53.975954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:54.013026 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431937b-5ac5-4601-aff9-7d1d7515e85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:54.013662 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431937b-5ac5-4601-aff9-7d1d7515e85a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:54.013942 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431937b-5ac5-4601-aff9-7d1d7515e85a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:54.014207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b431937b-5ac5-4601-aff9-7d1d7515e85a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:54.023706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431937b-5ac5-4601-aff9-7d1d7515e85a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:54.025149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b431937b-5ac5-4601-aff9-7d1d7515e85a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:54.037671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b431937b-5ac5-4601-aff9-7d1d7515e85a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:14:54.037671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:54.037671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:54.037671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:54.037671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:54.037671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:54.037671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:54.037671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:54.037671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:54.037671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:54.037671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:54.037671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:54.037671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:54.037671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:54.037671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:54.037671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:54.039844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:54.041383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:54.041383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:54.041383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:54.041383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:54.041383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:54.041383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:54.041383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:54.041383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:54.041383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:14:54.041383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:54.041383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 537/1071] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 15:14:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:54.045258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69b16e-43b2-4677-a560-94cba3361c7a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:54.155443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de69b16e-43b2-4677-a560-94cba3361c7a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:54.172413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:54.198001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e32e3-3c0b-413c-8034-87194a11b6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:54.198755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e32e3-3c0b-413c-8034-87194a11b6ca None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:54.199116 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e32e3-3c0b-413c-8034-87194a11b6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:54.199470 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e32e3-3c0b-413c-8034-87194a11b6ca None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:54.200285 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e32e3-3c0b-413c-8034-87194a11b6ca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:54.200683 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e32e3-3c0b-413c-8034-87194a11b6ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:54.202264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337e32e3-3c0b-413c-8034-87194a11b6ca None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:54.221234 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 535/1072] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 15:14:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:54.227907 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431937b-5ac5-4601-aff9-7d1d7515e85a None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:54.346811 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b431937b-5ac5-4601-aff9-7d1d7515e85a None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:54.393803 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ad9d9-d39e-4d7c-86d5-bec4ad92897b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:54.395624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ad9d9-d39e-4d7c-86d5-bec4ad92897b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:54.395963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ad9d9-d39e-4d7c-86d5-bec4ad92897b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:54.396302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ad9d9-d39e-4d7c-86d5-bec4ad92897b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:54.408629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ad9d9-d39e-4d7c-86d5-bec4ad92897b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:54.410096 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ad9d9-d39e-4d7c-86d5-bec4ad92897b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:54.425161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-359ad9d9-d39e-4d7c-86d5-bec4ad92897b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:54.425161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:54.425161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:54.425161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:54.425161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:54.425161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:54.425161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:54.425161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:54.425161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:54.425161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:54.425161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:54.425161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:54.425161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:54.425161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:54.425161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:54.425161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:54.426813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:54.428283 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:54.428283 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:54.428283 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:54.428283 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 538/1073] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:54.428913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b7288-69e0-4c65-8aa7-4297c53831c8 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:54.429208 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b7288-69e0-4c65-8aa7-4297c53831c8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:54.429483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971b7288-69e0-4c65-8aa7-4297c53831c8 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:54.431002 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 536/1074] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:14:54.433064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ad9d9-d39e-4d7c-86d5-bec4ad92897b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:54.437155 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47167-ea04-4aa8-9f87-1d7ecbb9ff2d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:54.437435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47167-ea04-4aa8-9f87-1d7ecbb9ff2d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:54.437796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef47167-ea04-4aa8-9f87-1d7ecbb9ff2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:54.459956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2ef47167-ea04-4aa8-9f87-1d7ecbb9ff2d None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:54.461167 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2ef47167-ea04-4aa8-9f87-1d7ecbb9ff2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:54.465406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:54.549305 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 537/1075] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:54.570203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ad9d9-d39e-4d7c-86d5-bec4ad92897b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:54.606679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc48e92-1c0e-4d21-ae0e-ad590f1688c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:54.607421 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc48e92-1c0e-4d21-ae0e-ad590f1688c9 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:54.607706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc48e92-1c0e-4d21-ae0e-ad590f1688c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:54.607983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc48e92-1c0e-4d21-ae0e-ad590f1688c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:54.608762 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc48e92-1c0e-4d21-ae0e-ad590f1688c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:54.609150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc48e92-1c0e-4d21-ae0e-ad590f1688c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:54.610541 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc48e92-1c0e-4d21-ae0e-ad590f1688c9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:54.623636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 539/1076] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:54.631710 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef47167-ea04-4aa8-9f87-1d7ecbb9ff2d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:54.732946 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:54.736846 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef47167-ea04-4aa8-9f87-1d7ecbb9ff2d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:54.771409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c82f6-7697-4996-908c-863b0486d02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:54.772026 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c82f6-7697-4996-908c-863b0486d02f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:54.772300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c82f6-7697-4996-908c-863b0486d02f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:54.772569 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c82f6-7697-4996-908c-863b0486d02f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:54.779214 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c82f6-7697-4996-908c-863b0486d02f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:54.780642 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c82f6-7697-4996-908c-863b0486d02f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:54.791538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-c63c82f6-7697-4996-908c-863b0486d02f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:54.791538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:54.791538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:54.791538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:54.791538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:54.791538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:54.791538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:54.791538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:54.791538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:54.791538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:54.791538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:54.791538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:54.791538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:54.791538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:54.791538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:54.792415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:54.793403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:54.793403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:54.793403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:54.793403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:54.793403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:54.793403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:54.793403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:54.793403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:54.793403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 538/1077] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:54.798951 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc48e92-1c0e-4d21-ae0e-ad590f1688c9 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:54.821839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c82f6-7697-4996-908c-863b0486d02f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:54.891292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:54.915623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc48e92-1c0e-4d21-ae0e-ad590f1688c9 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:54.927725 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c82f6-7697-4996-908c-863b0486d02f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:54.949183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89473712-d85c-43ab-bd7f-a1383af3f641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:54.949781 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89473712-d85c-43ab-bd7f-a1383af3f641 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:54.950056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89473712-d85c-43ab-bd7f-a1383af3f641 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:54.950356 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89473712-d85c-43ab-bd7f-a1383af3f641 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:54.951035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89473712-d85c-43ab-bd7f-a1383af3f641 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:54.951336 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89473712-d85c-43ab-bd7f-a1383af3f641 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:54.952741 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89473712-d85c-43ab-bd7f-a1383af3f641 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:54.962258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48380b-b979-4ef4-83bd-08f6bff1826c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:54.962983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48380b-b979-4ef4-83bd-08f6bff1826c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:54.963354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48380b-b979-4ef4-83bd-08f6bff1826c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:54.963714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d48380b-b979-4ef4-83bd-08f6bff1826c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:54.964880 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 540/1078] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:54.974584 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89473712-d85c-43ab-bd7f-a1383af3f641 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:54.978239 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d48380b-b979-4ef4-83bd-08f6bff1826c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:54.983632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-5d48380b-b979-4ef4-83bd-08f6bff1826c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:54.983632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:54.983632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:54.983632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:54.983632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:54.983632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:54.983632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:54.983632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:54.983632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:54.983632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:54.983632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:54.983632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:54.983632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:54.983632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:54.983632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:54.983632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:54.984547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:54.984547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:54.984547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:54.984547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:54.984547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:54.984547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:54.984547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:54.984547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:54.984547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:54.984547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:54.984547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:54.984547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:54.984547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:54.984547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:54.984547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:54.984547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:54.985390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:54.985390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:54.985390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:54.985390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:54.985390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:54.985390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:54.985390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:54.985390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:54.985390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:54.985390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:54.985390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:54.985390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:54.985390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 539/1079] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:14:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:54.990795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48380b-b979-4ef4-83bd-08f6bff1826c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:55.015180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:55.079130 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89473712-d85c-43ab-bd7f-a1383af3f641 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:55.098152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d48380b-b979-4ef4-83bd-08f6bff1826c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:55.114968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faed23c-56c4-4103-a4cc-1a249aeb3a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:55.115633 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faed23c-56c4-4103-a4cc-1a249aeb3a6b None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:55.115909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faed23c-56c4-4103-a4cc-1a249aeb3a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:55.116177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faed23c-56c4-4103-a4cc-1a249aeb3a6b None admin] PATH_INFO: `/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/8a5fd31a2db944f596912e4f0aa26835/roles/c87fe7f37cb246e8af3bc71057ec5048` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:55.118443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faed23c-56c4-4103-a4cc-1a249aeb3a6b None admin] RBAC: Authorizing `identity:create_grant(project_id=2c010096b77f4cd3b2f32086b1ba7ea7, user_id=8a5fd31a2db944f596912e4f0aa26835, role_id=c87fe7f37cb246e8af3bc71057ec5048)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:55.119874 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1faed23c-56c4-4103-a4cc-1a249aeb3a6b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:55.128814 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:55.135308 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 541/1080] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:14:54 2022] PUT /identity/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/8a5fd31a2db944f596912e4f0aa26835/roles/c87fe7f37cb246e8af3bc71057ec5048 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:14:55.157256 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033aa0b8-8876-40aa-8bf6-31feb87115b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:55.157803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033aa0b8-8876-40aa-8bf6-31feb87115b4 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:55.158109 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033aa0b8-8876-40aa-8bf6-31feb87115b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:55.158409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033aa0b8-8876-40aa-8bf6-31feb87115b4 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:55.159086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033aa0b8-8876-40aa-8bf6-31feb87115b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:55.159426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033aa0b8-8876-40aa-8bf6-31feb87115b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:55.160809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033aa0b8-8876-40aa-8bf6-31feb87115b4 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:55.171538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 540/1081] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:14:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:55.178463 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faed23c-56c4-4103-a4cc-1a249aeb3a6b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:55.311060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faed23c-56c4-4103-a4cc-1a249aeb3a6b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:55.347647 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43919f53-04bb-43e2-8614-822e18a352da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:55.348283 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43919f53-04bb-43e2-8614-822e18a352da None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:55.348560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43919f53-04bb-43e2-8614-822e18a352da None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:55.348830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43919f53-04bb-43e2-8614-822e18a352da None admin] PATH_INFO: `/v3/users/2f637462c68c4714bd92c97f7f05f72d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:55.349979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43919f53-04bb-43e2-8614-822e18a352da None admin] RBAC: Authorizing `identity:get_user(user_id=2f637462c68c4714bd92c97f7f05f72d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:55.351431 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43919f53-04bb-43e2-8614-822e18a352da None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:55.353022 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 542/1082] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:14:55 2022] GET /identity/v3/users/2f637462c68c4714bd92c97f7f05f72d => generated 285 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:55.358779 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033aa0b8-8876-40aa-8bf6-31feb87115b4 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:55.442964 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:55.461956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033aa0b8-8876-40aa-8bf6-31feb87115b4 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:55.498958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49480c31-dd1a-47d2-a3a0-db488c1eb6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:55.500002 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49480c31-dd1a-47d2-a3a0-db488c1eb6a1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:55.500489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49480c31-dd1a-47d2-a3a0-db488c1eb6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:55.500969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49480c31-dd1a-47d2-a3a0-db488c1eb6a1 None admin] PATH_INFO: `/v3/projects/c3e0ded9bb254e2888b04c1f0d31898d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:55.502893 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49480c31-dd1a-47d2-a3a0-db488c1eb6a1 None admin] RBAC: Authorizing `identity:get_project(project_id=c3e0ded9bb254e2888b04c1f0d31898d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:55.506044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49480c31-dd1a-47d2-a3a0-db488c1eb6a1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:55.509114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 541/1083] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:14:55 2022] GET /identity/v3/projects/c3e0ded9bb254e2888b04c1f0d31898d => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:55.515079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43919f53-04bb-43e2-8614-822e18a352da None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:55.588924 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:55.626918 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43919f53-04bb-43e2-8614-822e18a352da None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:55.663568 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d57bd3c-5cba-423b-a3d8-b24af625a6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:55.664220 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57bd3c-5cba-423b-a3d8-b24af625a6f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:55.664496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57bd3c-5cba-423b-a3d8-b24af625a6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:55.664764 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d57bd3c-5cba-423b-a3d8-b24af625a6f9 None admin] PATH_INFO: `/v3/projects/c3e0ded9bb254e2888b04c1f0d31898d/users/2f637462c68c4714bd92c97f7f05f72d/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:55.666704 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57bd3c-5cba-423b-a3d8-b24af625a6f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3e0ded9bb254e2888b04c1f0d31898d, user_id=2f637462c68c4714bd92c97f7f05f72d, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:55.668110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d57bd3c-5cba-423b-a3d8-b24af625a6f9 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:55.681182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 543/1084] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:14:55 2022] PUT /identity/v3/projects/c3e0ded9bb254e2888b04c1f0d31898d/users/2f637462c68c4714bd92c97f7f05f72d/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:14:57.048837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad316888-e734-413b-8b70-ea0684dc188b None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:57.049558 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad316888-e734-413b-8b70-ea0684dc188b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:57.049802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad316888-e734-413b-8b70-ea0684dc188b None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:57.052610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 542/1085] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:14:57.058326 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cd17e-7589-41cc-948f-dbb14e712f71 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:57.059112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cd17e-7589-41cc-948f-dbb14e712f71 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:57.059112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cd17e-7589-41cc-948f-dbb14e712f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:57.077654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-eb3cd17e-7589-41cc-948f-dbb14e712f71 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:57.078859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-eb3cd17e-7589-41cc-948f-dbb14e712f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:57.156684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 544/1086] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:57.420676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad316888-e734-413b-8b70-ea0684dc188b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:57.538505 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad316888-e734-413b-8b70-ea0684dc188b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:57.574984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26330958-0af0-451d-99ab-fdd5b83679c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:57.575611 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26330958-0af0-451d-99ab-fdd5b83679c1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:57.575890 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26330958-0af0-451d-99ab-fdd5b83679c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:57.576160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26330958-0af0-451d-99ab-fdd5b83679c1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:57.588113 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26330958-0af0-451d-99ab-fdd5b83679c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:57.592925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-26330958-0af0-451d-99ab-fdd5b83679c1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:14:57.592925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:57.592925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:57.592925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:57.592925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:57.592925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:57.592925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:57.592925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:57.592925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:57.592925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:57.592925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:57.592925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:57.592925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:57.592925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:57.592925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:57.592925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:57.594127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:57.594127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:57.594127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:57.594127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:57.594127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:57.594127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:57.594127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:57.594127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:57.594127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:57.594127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:57.594127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:57.594127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:57.594127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:57.594127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:57.594127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:57.594127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:57.595263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:57.595263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:57.595263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:57.595263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:57.595263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:57.595263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:57.595263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:57.595263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:57.595263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:57.595263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:57.595263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 15:14:57.595263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:57.598362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 543/1087] 10.222.0.76 () {56 vars in 1120 bytes} [Wed Aug 24 15:14:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:57.599917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cd17e-7589-41cc-948f-dbb14e712f71 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:57.615686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bf4df-e376-4a66-b89d-2d43e34e34cd None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:57.616011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bf4df-e376-4a66-b89d-2d43e34e34cd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:57.616322 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8bf4df-e376-4a66-b89d-2d43e34e34cd None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:57.617675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 544/1088] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:14:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:14:57.623145 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58de23-2392-4c62-8e65-ceba595a27b4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:57.623555 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58de23-2392-4c62-8e65-ceba595a27b4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:57.623928 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a58de23-2392-4c62-8e65-ceba595a27b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:57.642424 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6a58de23-2392-4c62-8e65-ceba595a27b4 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:14:57.643610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6a58de23-2392-4c62-8e65-ceba595a27b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:14:57.647793 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:57.661543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:57.707196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cd17e-7589-41cc-948f-dbb14e712f71 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:57.713974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 545/1089] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:14:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:14:57.742288 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5a1cd-433a-4681-97f7-01bf3d13c806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:57.742889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5a1cd-433a-4681-97f7-01bf3d13c806 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:57.743168 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5a1cd-433a-4681-97f7-01bf3d13c806 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:57.743440 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5a1cd-433a-4681-97f7-01bf3d13c806 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:57.744105 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5a1cd-433a-4681-97f7-01bf3d13c806 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:57.744405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5a1cd-433a-4681-97f7-01bf3d13c806 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:57.745877 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df5a1cd-433a-4681-97f7-01bf3d13c806 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:57.754330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 545/1090] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 24 15:14:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:57.759893 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58de23-2392-4c62-8e65-ceba595a27b4 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:57.861357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a58de23-2392-4c62-8e65-ceba595a27b4 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:57.900946 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d74890-b116-419d-a606-3a1f27b2e44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:57.901823 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d74890-b116-419d-a606-3a1f27b2e44e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:57.902104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d74890-b116-419d-a606-3a1f27b2e44e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:57.902548 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d74890-b116-419d-a606-3a1f27b2e44e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:57.911543 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:57.913866 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d74890-b116-419d-a606-3a1f27b2e44e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:57.915865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d74890-b116-419d-a606-3a1f27b2e44e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:57.930921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-15d74890-b116-419d-a606-3a1f27b2e44e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:57.930921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:57.930921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:57.930921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:57.930921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:57.930921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:57.930921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:57.930921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:57.930921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:57.930921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:57.930921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:57.930921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:57.930921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:57.930921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:57.930921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:57.930921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:57.932094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:57.933002 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:57.933002 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:57.933002 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:57.933002 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 546/1091] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:57.938005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5a1cd-433a-4681-97f7-01bf3d13c806 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:57.982581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d74890-b116-419d-a606-3a1f27b2e44e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:58.040106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5a1cd-433a-4681-97f7-01bf3d13c806 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:58.057825 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:58.076329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33924e86-3cd7-4a34-a5d7-06fcb079ab63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:58.076973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33924e86-3cd7-4a34-a5d7-06fcb079ab63 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:58.077195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33924e86-3cd7-4a34-a5d7-06fcb079ab63 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:58.077467 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33924e86-3cd7-4a34-a5d7-06fcb079ab63 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:58.078102 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33924e86-3cd7-4a34-a5d7-06fcb079ab63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:58.078433 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33924e86-3cd7-4a34-a5d7-06fcb079ab63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:58.079783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33924e86-3cd7-4a34-a5d7-06fcb079ab63 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:58.085251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d74890-b116-419d-a606-3a1f27b2e44e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:58.092198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 546/1092] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:58.098376 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33924e86-3cd7-4a34-a5d7-06fcb079ab63 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:58.120656 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5bb87-9c13-4862-8b1c-5de99f49f636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:58.121280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5bb87-9c13-4862-8b1c-5de99f49f636 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:58.121554 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5bb87-9c13-4862-8b1c-5de99f49f636 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:58.121821 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd5bb87-9c13-4862-8b1c-5de99f49f636 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:58.132949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd5bb87-9c13-4862-8b1c-5de99f49f636 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:14:58.137657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-0cd5bb87-9c13-4862-8b1c-5de99f49f636 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:58.137657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:58.137657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:58.137657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:58.137657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:58.137657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:58.137657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:58.137657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:58.137657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:58.137657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:58.137657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:58.137657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:58.137657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:14:58.137657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:14:58.137657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:14:58.137657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:14:58.138984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:14:58.138984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:14:58.138984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:14:58.138984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:14:58.138984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:58.138984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:58.138984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:58.138984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:58.138984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:58.138984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:58.138984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:58.138984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:58.138984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:58.138984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:58.138984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:58.138984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:58.140257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:58.140257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:58.140257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:58.140257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:58.140257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:14:58.140257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:14:58.140257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:14:58.140257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:14:58.140257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:14:58.140257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:14:58.140257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:14:58.140257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:58.140257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 547/1093] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:14:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:58.145304 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5bb87-9c13-4862-8b1c-5de99f49f636 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:58.197186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:58.201721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33924e86-3cd7-4a34-a5d7-06fcb079ab63 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:58.236365 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58c12a-baf1-4cc9-b0e6-03e7efdbb720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:58.236965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58c12a-baf1-4cc9-b0e6-03e7efdbb720 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:58.237238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58c12a-baf1-4cc9-b0e6-03e7efdbb720 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:58.237547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58c12a-baf1-4cc9-b0e6-03e7efdbb720 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:58.244633 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58c12a-baf1-4cc9-b0e6-03e7efdbb720 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:58.245703 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd5bb87-9c13-4862-8b1c-5de99f49f636 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:58.246068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed58c12a-baf1-4cc9-b0e6-03e7efdbb720 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:58.258149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-ed58c12a-baf1-4cc9-b0e6-03e7efdbb720 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:14:58.258149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:58.258149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:58.258149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:58.258149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:58.258149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:58.258149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:58.258149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:58.258149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:58.258149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:58.258149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:58.258149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:58.258149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:14:58.258149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:14:58.258149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:14:58.258149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:58.259340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:58.260730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:58.260730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:58.260730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:14:58.260730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:14:58.260730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:14:58.260730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:14:58.260730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:14:58.260730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:14:58.260730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 15:14:58.260730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:58.260730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 547/1094] 10.222.0.76 () {56 vars in 1093 bytes} [Wed Aug 24 15:14:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:58.264955 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58c12a-baf1-4cc9-b0e6-03e7efdbb720 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:58.286332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46595738-1565-4c37-883b-fb192048d887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:58.286997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46595738-1565-4c37-883b-fb192048d887 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:58.287423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46595738-1565-4c37-883b-fb192048d887 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:58.287818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46595738-1565-4c37-883b-fb192048d887 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:58.288613 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46595738-1565-4c37-883b-fb192048d887 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:58.288977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46595738-1565-4c37-883b-fb192048d887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:58.290435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46595738-1565-4c37-883b-fb192048d887 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:58.299611 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 548/1095] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:14:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:58.305414 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46595738-1565-4c37-883b-fb192048d887 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:58.309688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:58.374797 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed58c12a-baf1-4cc9-b0e6-03e7efdbb720 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:58.406957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46595738-1565-4c37-883b-fb192048d887 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:58.412309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86af6ea-9d45-4cca-a46e-392eaae410c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:58.412917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86af6ea-9d45-4cca-a46e-392eaae410c2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:58.413197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86af6ea-9d45-4cca-a46e-392eaae410c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:58.413468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86af6ea-9d45-4cca-a46e-392eaae410c2 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:58.414197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86af6ea-9d45-4cca-a46e-392eaae410c2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:58.414534 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86af6ea-9d45-4cca-a46e-392eaae410c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:58.415948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86af6ea-9d45-4cca-a46e-392eaae410c2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:58.421594 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:58.437133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 548/1096] 10.222.0.76 () {56 vars in 1133 bytes} [Wed Aug 24 15:14:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:58.443301 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86af6ea-9d45-4cca-a46e-392eaae410c2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:58.443992 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa56bc5-8c1a-4edf-be55-c509c19a248c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:58.444649 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa56bc5-8c1a-4edf-be55-c509c19a248c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:58.444925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa56bc5-8c1a-4edf-be55-c509c19a248c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:58.445194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa56bc5-8c1a-4edf-be55-c509c19a248c None admin] PATH_INFO: `/v3/users/2f637462c68c4714bd92c97f7f05f72d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:58.446381 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa56bc5-8c1a-4edf-be55-c509c19a248c None admin] RBAC: Authorizing `identity:get_user(user_id=2f637462c68c4714bd92c97f7f05f72d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:58.448128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa56bc5-8c1a-4edf-be55-c509c19a248c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:58.454597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 549/1097] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:14:58 2022] GET /identity/v3/users/2f637462c68c4714bd92c97f7f05f72d => generated 285 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:58.455540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa56bc5-8c1a-4edf-be55-c509c19a248c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:58.544438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:58.551711 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86af6ea-9d45-4cca-a46e-392eaae410c2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:58.559529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa56bc5-8c1a-4edf-be55-c509c19a248c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:58.586596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f4a67-03ac-415a-94c5-eb62bebc4a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:58.587204 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f4a67-03ac-415a-94c5-eb62bebc4a73 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:58.587482 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f4a67-03ac-415a-94c5-eb62bebc4a73 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:58.587752 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2f4a67-03ac-415a-94c5-eb62bebc4a73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:58.595686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81cc1ba-cdcd-41bc-a459-856cbb649e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:58.596279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cc1ba-cdcd-41bc-a459-856cbb649e37 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:58.596582 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cc1ba-cdcd-41bc-a459-856cbb649e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:58.597421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81cc1ba-cdcd-41bc-a459-856cbb649e37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:58.650871 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:58.663849 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f4a67-03ac-415a-94c5-eb62bebc4a73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:58.665298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2f4a67-03ac-415a-94c5-eb62bebc4a73 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:58.674139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81cc1ba-cdcd-41bc-a459-856cbb649e37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:58.675724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81cc1ba-cdcd-41bc-a459-856cbb649e37 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:58.777532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:58.790261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 549/1098] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:14:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 15:14:58.792302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f4a67-03ac-415a-94c5-eb62bebc4a73 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:58.799636 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 550/1099] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:14:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 15:14:58.805976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81cc1ba-cdcd-41bc-a459-856cbb649e37 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:58.912567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:58.915552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2f4a67-03ac-415a-94c5-eb62bebc4a73 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:58.932980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81cc1ba-cdcd-41bc-a459-856cbb649e37 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:58.956165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917e5ce-ee95-4f68-820b-a644b5f140ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:58.957562 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917e5ce-ee95-4f68-820b-a644b5f140ac None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:58.957562 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917e5ce-ee95-4f68-820b-a644b5f140ac None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:58.957562 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917e5ce-ee95-4f68-820b-a644b5f140ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:58.970324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917e5ce-ee95-4f68-820b-a644b5f140ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:58.972278 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917e5ce-ee95-4f68-820b-a644b5f140ac None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:58.976591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb6ffc-9aa0-49ee-9ebd-87d2e21bc776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:58.977577 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb6ffc-9aa0-49ee-9ebd-87d2e21bc776 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:58.977865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb6ffc-9aa0-49ee-9ebd-87d2e21bc776 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:58.978138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb6ffc-9aa0-49ee-9ebd-87d2e21bc776 None admin] PATH_INFO: `/v3/projects/c3e0ded9bb254e2888b04c1f0d31898d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:58.979434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb6ffc-9aa0-49ee-9ebd-87d2e21bc776 None admin] RBAC: Authorizing `identity:get_project(project_id=c3e0ded9bb254e2888b04c1f0d31898d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:58.980862 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb6ffc-9aa0-49ee-9ebd-87d2e21bc776 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:58.982582 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 551/1100] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:14:58 2022] GET /identity/v3/projects/c3e0ded9bb254e2888b04c1f0d31898d => generated 311 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:58.988640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-5917e5ce-ee95-4f68-820b-a644b5f140ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:58.988640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:58.988640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:58.988640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:58.988640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:58.988640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:58.988640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:58.988640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:58.988640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:58.988640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:58.988640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:58.988640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:58.988640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:14:58.988640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:14:58.988640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:14:58.988640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:58.989822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:14:58.990764 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:14:58.990764 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:14:58.990764 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:14:58.990947 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb6ffc-9aa0-49ee-9ebd-87d2e21bc776 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:58.991020 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 550/1101] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:14:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:14:58.996927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917e5ce-ee95-4f68-820b-a644b5f140ac None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:59.057992 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:59.105887 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb6ffc-9aa0-49ee-9ebd-87d2e21bc776 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:59.122829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917e5ce-ee95-4f68-820b-a644b5f140ac None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:59.146891 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8761ff-397c-41c5-8c4c-19abfb11d460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:59.147490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8761ff-397c-41c5-8c4c-19abfb11d460 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:59.147764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8761ff-397c-41c5-8c4c-19abfb11d460 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:59.148037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8761ff-397c-41c5-8c4c-19abfb11d460 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:59.148756 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8761ff-397c-41c5-8c4c-19abfb11d460 None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=c3e0ded9bb254e2888b04c1f0d31898d, user.id=2f637462c68c4714bd92c97f7f05f72d) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:59.149071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8761ff-397c-41c5-8c4c-19abfb11d460 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:59.150515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8761ff-397c-41c5-8c4c-19abfb11d460 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:59.156962 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 552/1102] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:14:58 2022] GET /identity/v3/role_assignments?user.id=2f637462c68c4714bd92c97f7f05f72d&scope.project.id=c3e0ded9bb254e2888b04c1f0d31898d&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:14:59.165116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59b99f-5a20-4563-8e89-a895a17a7df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:59.165748 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59b99f-5a20-4563-8e89-a895a17a7df7 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:59.166025 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59b99f-5a20-4563-8e89-a895a17a7df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:59.166327 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e59b99f-5a20-4563-8e89-a895a17a7df7 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:59.166992 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59b99f-5a20-4563-8e89-a895a17a7df7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:59.167298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59b99f-5a20-4563-8e89-a895a17a7df7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:59.168702 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e59b99f-5a20-4563-8e89-a895a17a7df7 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:59.185580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 551/1103] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:14:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:59.194416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8761ff-397c-41c5-8c4c-19abfb11d460 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:59.213213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:59.314413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8761ff-397c-41c5-8c4c-19abfb11d460 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:59.360960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496c166-7309-44c0-93ea-1936e0380153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:59.361592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496c166-7309-44c0-93ea-1936e0380153 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:59.361867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496c166-7309-44c0-93ea-1936e0380153 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:59.362142 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7496c166-7309-44c0-93ea-1936e0380153 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:59.369939 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496c166-7309-44c0-93ea-1936e0380153 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:59.371430 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7496c166-7309-44c0-93ea-1936e0380153 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:59.383586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-7496c166-7309-44c0-93ea-1936e0380153 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:59.383586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:14:59.383586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:14:59.383586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:14:59.383586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:14:59.383586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:14:59.383586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:14:59.383586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:14:59.383586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:14:59.383586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:14:59.383586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:14:59.383586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:14:59.383586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:14:59.383586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:14:59.383586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:14:59.384584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:14:59.385698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:14:59.385698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:14:59.385698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:14:59.385698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:14:59.385698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:14:59.385698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:14:59.385698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:14:59.385698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:14:59.385698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 553/1104] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:14:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:14:59.390700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59b99f-5a20-4563-8e89-a895a17a7df7 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:59.504942 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e59b99f-5a20-4563-8e89-a895a17a7df7 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:59.515242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:59.559331 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7d7e25-48c8-4dfa-8c04-19124121a933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:59.560265 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d7e25-48c8-4dfa-8c04-19124121a933 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:59.560746 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d7e25-48c8-4dfa-8c04-19124121a933 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:59.561207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7d7e25-48c8-4dfa-8c04-19124121a933 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:59.562502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d7e25-48c8-4dfa-8c04-19124121a933 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:59.563047 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d7e25-48c8-4dfa-8c04-19124121a933 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:59.566413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7d7e25-48c8-4dfa-8c04-19124121a933 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:59.594684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 552/1105] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:14:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:14:59.596488 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496c166-7309-44c0-93ea-1936e0380153 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:14:59.679366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:14:59.708635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7496c166-7309-44c0-93ea-1936e0380153 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:14:59.751436 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27c42b-78b9-484f-9f79-f8c21c8c234b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:14:59.752038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27c42b-78b9-484f-9f79-f8c21c8c234b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:14:59.752308 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27c42b-78b9-484f-9f79-f8c21c8c234b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:14:59.752601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27c42b-78b9-484f-9f79-f8c21c8c234b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:14:59.753894 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27c42b-78b9-484f-9f79-f8c21c8c234b None admin] RBAC: Adding query filter params (role.id=c87fe7f37cb246e8af3bc71057ec5048, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=8a5fd31a2db944f596912e4f0aa26835) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:14:59.754197 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27c42b-78b9-484f-9f79-f8c21c8c234b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:14:59.755769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27c42b-78b9-484f-9f79-f8c21c8c234b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:14:59.762289 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 554/1106] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:14:59 2022] GET /identity/v3/role_assignments?user.id=8a5fd31a2db944f596912e4f0aa26835&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=c87fe7f37cb246e8af3bc71057ec5048 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:01.173535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339444f-0925-47e8-a2a1-642da89c3678 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:01.174078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339444f-0925-47e8-a2a1-642da89c3678 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:01.174078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339444f-0925-47e8-a2a1-642da89c3678 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:01.176119 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 553/1107] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:15:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:15:01.181865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f9bb9-da59-4173-b6e6-b1663eb62af4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:01.182566 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f9bb9-da59-4173-b6e6-b1663eb62af4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:01.182566 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8f9bb9-da59-4173-b6e6-b1663eb62af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:01.200832 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8b8f9bb9-da59-4173-b6e6-b1663eb62af4 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:15:01.202050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8b8f9bb9-da59-4173-b6e6-b1663eb62af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:15:01.276757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 555/1108] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:15:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:15:01.546839 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339444f-0925-47e8-a2a1-642da89c3678 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:01.647747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339444f-0925-47e8-a2a1-642da89c3678 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:01.680921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ece4b5-953a-4cea-b99b-6822471c368e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:01.681535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ece4b5-953a-4cea-b99b-6822471c368e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:01.681811 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ece4b5-953a-4cea-b99b-6822471c368e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:01.682079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ece4b5-953a-4cea-b99b-6822471c368e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:01.683362 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ece4b5-953a-4cea-b99b-6822471c368e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:01.684775 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ece4b5-953a-4cea-b99b-6822471c368e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:01.686352 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 554/1109] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:15:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:01.692215 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f9bb9-da59-4173-b6e6-b1663eb62af4 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:01.758028 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:01.791538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8f9bb9-da59-4173-b6e6-b1663eb62af4 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:01.824020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89619c5-380f-4e53-afb4-1e3f48c42d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:01.824626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89619c5-380f-4e53-afb4-1e3f48c42d2e None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:01.824903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89619c5-380f-4e53-afb4-1e3f48c42d2e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:01.825172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89619c5-380f-4e53-afb4-1e3f48c42d2e None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:01.825890 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89619c5-380f-4e53-afb4-1e3f48c42d2e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:01.827334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89619c5-380f-4e53-afb4-1e3f48c42d2e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:01.848793 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 556/1110] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:15:01 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:15:03.766108 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34763d-21bc-40cd-90a3-ef5494e74211 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:03.766699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34763d-21bc-40cd-90a3-ef5494e74211 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:03.766699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34763d-21bc-40cd-90a3-ef5494e74211 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:03.768016 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 555/1111] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:15:03.773225 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee1f99-f6e8-4449-af3e-c38869036658 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:03.773664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee1f99-f6e8-4449-af3e-c38869036658 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:03.773757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee1f99-f6e8-4449-af3e-c38869036658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:03.792339 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ffee1f99-f6e8-4449-af3e-c38869036658 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:15:03.793292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ffee1f99-f6e8-4449-af3e-c38869036658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:15:03.858455 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 557/1112] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:15:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:15:04.132671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34763d-21bc-40cd-90a3-ef5494e74211 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:04.173384 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:04.251321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34763d-21bc-40cd-90a3-ef5494e74211 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:04.290889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6393316-d95c-4804-8095-9179d956e4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:04.291514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6393316-d95c-4804-8095-9179d956e4ac None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:04.291793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6393316-d95c-4804-8095-9179d956e4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:04.292060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6393316-d95c-4804-8095-9179d956e4ac None admin] PATH_INFO: `/v3/roles/d123411b649d4acb8648d0c0b80aae6d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:04.293477 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6393316-d95c-4804-8095-9179d956e4ac None admin] RBAC: Authorizing `identity:get_role(role_id=d123411b649d4acb8648d0c0b80aae6d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:04.294874 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6393316-d95c-4804-8095-9179d956e4ac None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:04.296199 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 556/1113] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:15:04 2022] GET /identity/v3/roles/d123411b649d4acb8648d0c0b80aae6d => generated 223 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:04.302874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee1f99-f6e8-4449-af3e-c38869036658 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:04.357615 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:04.411270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee1f99-f6e8-4449-af3e-c38869036658 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:04.451328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da566d4-5aed-4f62-b2e0-b62e583f4d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:04.451980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da566d4-5aed-4f62-b2e0-b62e583f4d8f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:04.452262 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da566d4-5aed-4f62-b2e0-b62e583f4d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:04.452536 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da566d4-5aed-4f62-b2e0-b62e583f4d8f None admin] PATH_INFO: `/v3/users/8298369a0770410086517e9de2031ba8` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:04.467170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da566d4-5aed-4f62-b2e0-b62e583f4d8f None admin] RBAC: Authorizing `identity:get_user(user_id=8298369a0770410086517e9de2031ba8)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:04.468666 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da566d4-5aed-4f62-b2e0-b62e583f4d8f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:04.470425 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 558/1114] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:15:04 2022] GET /identity/v3/users/8298369a0770410086517e9de2031ba8 => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:15:04.477006 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6393316-d95c-4804-8095-9179d956e4ac None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:04.585974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6393316-d95c-4804-8095-9179d956e4ac None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:04.588462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:04.621911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b3bac-cf67-4684-9bb4-8af3dbcbdc66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:04.622551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b3bac-cf67-4684-9bb4-8af3dbcbdc66 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:04.622833 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b3bac-cf67-4684-9bb4-8af3dbcbdc66 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:04.623110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b3bac-cf67-4684-9bb4-8af3dbcbdc66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:04.690792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b3bac-cf67-4684-9bb4-8af3dbcbdc66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:04.692246 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b3bac-cf67-4684-9bb4-8af3dbcbdc66 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:04.788552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 557/1115] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:15:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:15:04.798914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da566d4-5aed-4f62-b2e0-b62e583f4d8f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:04.897873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da566d4-5aed-4f62-b2e0-b62e583f4d8f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:04.910986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:04.932058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc420c7-7cd6-495f-bb21-bb0202bd1ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:04.932669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc420c7-7cd6-495f-bb21-bb0202bd1ddf None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:04.932944 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc420c7-7cd6-495f-bb21-bb0202bd1ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:04.933215 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc420c7-7cd6-495f-bb21-bb0202bd1ddf None admin] PATH_INFO: `/v3/projects/c3e0ded9bb254e2888b04c1f0d31898d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:04.934329 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc420c7-7cd6-495f-bb21-bb0202bd1ddf None admin] RBAC: Authorizing `identity:get_project(project_id=c3e0ded9bb254e2888b04c1f0d31898d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:04.935742 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc420c7-7cd6-495f-bb21-bb0202bd1ddf None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:04.937342 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 559/1116] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:15:04 2022] GET /identity/v3/projects/c3e0ded9bb254e2888b04c1f0d31898d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:04.944137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b3bac-cf67-4684-9bb4-8af3dbcbdc66 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:04.947176 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:05.041191 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b3bac-cf67-4684-9bb4-8af3dbcbdc66 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:05.079737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2da8595-130b-424e-99f7-5e8ecba55f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:05.080322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da8595-130b-424e-99f7-5e8ecba55f7b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:05.080594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da8595-130b-424e-99f7-5e8ecba55f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:05.080901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2da8595-130b-424e-99f7-5e8ecba55f7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:05.082098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da8595-130b-424e-99f7-5e8ecba55f7b None admin] RBAC: Adding query filter params (role.id=d123411b649d4acb8648d0c0b80aae6d, scope.project.id=c3e0ded9bb254e2888b04c1f0d31898d, user.id=8298369a0770410086517e9de2031ba8) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:15:05.082431 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da8595-130b-424e-99f7-5e8ecba55f7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:05.083836 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2da8595-130b-424e-99f7-5e8ecba55f7b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:05.089111 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 558/1117] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:15:04 2022] GET /identity/v3/role_assignments?user.id=8298369a0770410086517e9de2031ba8&scope.project.id=c3e0ded9bb254e2888b04c1f0d31898d&role.id=d123411b649d4acb8648d0c0b80aae6d => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:06.980984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d3206-d0bb-4301-92f6-2cd0765914da None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:06.981553 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d3206-d0bb-4301-92f6-2cd0765914da None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:06.981553 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1d3206-d0bb-4301-92f6-2cd0765914da None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:06.982881 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 560/1118] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:15:06.988529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcd2e8-f265-46ab-9148-613f8c111e94 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:06.988937 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcd2e8-f265-46ab-9148-613f8c111e94 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:06.989083 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fcd2e8-f265-46ab-9148-613f8c111e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:07.007518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-61fcd2e8-f265-46ab-9148-613f8c111e94 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:15:07.008728 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-61fcd2e8-f265-46ab-9148-613f8c111e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:15:07.076423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:07.082816 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 559/1119] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:15:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:15:07.354925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d3206-d0bb-4301-92f6-2cd0765914da None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:07.455647 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1d3206-d0bb-4301-92f6-2cd0765914da None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:07.493637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94c721-ba6c-477c-add5-6b06be99c793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:07.494276 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94c721-ba6c-477c-add5-6b06be99c793 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:07.494550 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94c721-ba6c-477c-add5-6b06be99c793 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:07.494814 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94c721-ba6c-477c-add5-6b06be99c793 None admin] PATH_INFO: `/v3/roles/d123411b649d4acb8648d0c0b80aae6d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:07.496329 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94c721-ba6c-477c-add5-6b06be99c793 None admin] RBAC: Authorizing `identity:get_role(role_id=d123411b649d4acb8648d0c0b80aae6d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:07.497693 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d94c721-ba6c-477c-add5-6b06be99c793 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:07.499052 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 561/1120] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:15:07 2022] GET /identity/v3/roles/d123411b649d4acb8648d0c0b80aae6d => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:07.504943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcd2e8-f265-46ab-9148-613f8c111e94 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:07.606884 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fcd2e8-f265-46ab-9148-613f8c111e94 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:07.641492 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4a66d-26ec-4bba-852f-f9de8e98e959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:07.642120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4a66d-26ec-4bba-852f-f9de8e98e959 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:07.642427 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4a66d-26ec-4bba-852f-f9de8e98e959 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:07.642700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c4a66d-26ec-4bba-852f-f9de8e98e959 None admin] PATH_INFO: `/v3/users/8298369a0770410086517e9de2031ba8` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:07.643805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4a66d-26ec-4bba-852f-f9de8e98e959 None admin] RBAC: Authorizing `identity:get_user(user_id=8298369a0770410086517e9de2031ba8)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:07.645265 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c4a66d-26ec-4bba-852f-f9de8e98e959 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:07.646883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 560/1121] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:15:07 2022] GET /identity/v3/users/8298369a0770410086517e9de2031ba8 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:07.653242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94c721-ba6c-477c-add5-6b06be99c793 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:07.703974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:07.754515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94c721-ba6c-477c-add5-6b06be99c793 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:07.787833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f77688a-cbbf-4b18-9e20-6e009f305d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:07.788504 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77688a-cbbf-4b18-9e20-6e009f305d8a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:07.788800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77688a-cbbf-4b18-9e20-6e009f305d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:07.789066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f77688a-cbbf-4b18-9e20-6e009f305d8a None admin] PATH_INFO: `/v3/projects/c3e0ded9bb254e2888b04c1f0d31898d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:07.790155 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77688a-cbbf-4b18-9e20-6e009f305d8a None admin] RBAC: Authorizing `identity:get_project(project_id=c3e0ded9bb254e2888b04c1f0d31898d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:07.791596 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f77688a-cbbf-4b18-9e20-6e009f305d8a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:07.793223 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 562/1122] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:15:07 2022] GET /identity/v3/projects/c3e0ded9bb254e2888b04c1f0d31898d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:07.799367 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4a66d-26ec-4bba-852f-f9de8e98e959 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:07.866563 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:07.899904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c4a66d-26ec-4bba-852f-f9de8e98e959 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:07.934434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c89ed9-4305-4662-8c09-ae086629de31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:07.935080 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c89ed9-4305-4662-8c09-ae086629de31 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:07.935352 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c89ed9-4305-4662-8c09-ae086629de31 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:07.935620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c89ed9-4305-4662-8c09-ae086629de31 None admin] PATH_INFO: `/v3/projects/c3e0ded9bb254e2888b04c1f0d31898d/users/8298369a0770410086517e9de2031ba8/roles/d123411b649d4acb8648d0c0b80aae6d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:07.937744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c89ed9-4305-4662-8c09-ae086629de31 None admin] RBAC: Authorizing `identity:create_grant(project_id=c3e0ded9bb254e2888b04c1f0d31898d, user_id=8298369a0770410086517e9de2031ba8, role_id=d123411b649d4acb8648d0c0b80aae6d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:07.939190 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c89ed9-4305-4662-8c09-ae086629de31 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:07.951654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 561/1123] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:15:07 2022] PUT /identity/v3/projects/c3e0ded9bb254e2888b04c1f0d31898d/users/8298369a0770410086517e9de2031ba8/roles/d123411b649d4acb8648d0c0b80aae6d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:15:09.916926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431a5f6-70bc-481c-b76b-d87e69144532 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:09.917457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431a5f6-70bc-481c-b76b-d87e69144532 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:09.917457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c431a5f6-70bc-481c-b76b-d87e69144532 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:09.918839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 563/1124] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:15:09.924078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f0592-c85a-4c24-b3fe-5fbd308ce14c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:09.924522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f0592-c85a-4c24-b3fe-5fbd308ce14c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:09.924661 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f0592-c85a-4c24-b3fe-5fbd308ce14c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:09.942415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-673f0592-c85a-4c24-b3fe-5fbd308ce14c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:15:09.943607 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-673f0592-c85a-4c24-b3fe-5fbd308ce14c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:15:10.009970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 562/1125] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:15:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:15:10.285075 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431a5f6-70bc-481c-b76b-d87e69144532 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:10.388563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c431a5f6-70bc-481c-b76b-d87e69144532 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:10.405974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:10.424634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf5785-abdb-427e-a620-127228104f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:10.425254 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf5785-abdb-427e-a620-127228104f82 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:10.425535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf5785-abdb-427e-a620-127228104f82 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:10.425806 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cf5785-abdb-427e-a620-127228104f82 None admin] PATH_INFO: `/v3/roles/d123411b649d4acb8648d0c0b80aae6d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:10.427204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf5785-abdb-427e-a620-127228104f82 None admin] RBAC: Authorizing `identity:get_role(role_id=d123411b649d4acb8648d0c0b80aae6d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:10.428575 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cf5785-abdb-427e-a620-127228104f82 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:10.429902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 564/1126] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:15:10 2022] GET /identity/v3/roles/d123411b649d4acb8648d0c0b80aae6d => generated 223 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:10.435927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f0592-c85a-4c24-b3fe-5fbd308ce14c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:10.501324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:10.546584 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673f0592-c85a-4c24-b3fe-5fbd308ce14c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:10.582592 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33028a-af20-42e2-842b-62902fbd02df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:10.583221 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33028a-af20-42e2-842b-62902fbd02df None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:10.583498 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33028a-af20-42e2-842b-62902fbd02df None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:10.583770 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f33028a-af20-42e2-842b-62902fbd02df None admin] PATH_INFO: `/v3/users/8298369a0770410086517e9de2031ba8` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:10.584885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33028a-af20-42e2-842b-62902fbd02df None admin] RBAC: Authorizing `identity:get_user(user_id=8298369a0770410086517e9de2031ba8)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:10.586363 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f33028a-af20-42e2-842b-62902fbd02df None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:10.587933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 563/1127] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:15:10 2022] GET /identity/v3/users/8298369a0770410086517e9de2031ba8 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:10.597423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf5785-abdb-427e-a620-127228104f82 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:10.697518 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cf5785-abdb-427e-a620-127228104f82 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:10.731083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2837ec2-aca4-4a32-ba87-01abc740d618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:10.731683 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2837ec2-aca4-4a32-ba87-01abc740d618 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:10.731964 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2837ec2-aca4-4a32-ba87-01abc740d618 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:10.732236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2837ec2-aca4-4a32-ba87-01abc740d618 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:10.783470 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:10.808920 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2837ec2-aca4-4a32-ba87-01abc740d618 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:10.810344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2837ec2-aca4-4a32-ba87-01abc740d618 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:10.907209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 565/1128] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:15:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:15:10.914038 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33028a-af20-42e2-842b-62902fbd02df None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:11.013865 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f33028a-af20-42e2-842b-62902fbd02df None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:11.037297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:11.048102 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44fcfd1-ce38-47e3-90cb-2ed2449b717f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:11.048731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fcfd1-ce38-47e3-90cb-2ed2449b717f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:11.048999 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fcfd1-ce38-47e3-90cb-2ed2449b717f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:11.049299 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44fcfd1-ce38-47e3-90cb-2ed2449b717f None admin] PATH_INFO: `/v3/projects/c3e0ded9bb254e2888b04c1f0d31898d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:11.051011 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44fcfd1-ce38-47e3-90cb-2ed2449b717f None admin] RBAC: Authorizing `identity:get_project(project_id=c3e0ded9bb254e2888b04c1f0d31898d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:11.052423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44fcfd1-ce38-47e3-90cb-2ed2449b717f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:11.054084 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 564/1129] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:15:10 2022] GET /identity/v3/projects/c3e0ded9bb254e2888b04c1f0d31898d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:11.060540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2837ec2-aca4-4a32-ba87-01abc740d618 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:11.150538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:11.166856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2837ec2-aca4-4a32-ba87-01abc740d618 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:11.202013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed429fd-e589-4837-8df4-93a479cbafe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:11.202669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed429fd-e589-4837-8df4-93a479cbafe3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:11.202943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed429fd-e589-4837-8df4-93a479cbafe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:11.203207 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed429fd-e589-4837-8df4-93a479cbafe3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:11.203915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed429fd-e589-4837-8df4-93a479cbafe3 None admin] RBAC: Adding query filter params (role.id=d123411b649d4acb8648d0c0b80aae6d, scope.project.id=c3e0ded9bb254e2888b04c1f0d31898d, user.id=8298369a0770410086517e9de2031ba8) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:15:11.204214 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed429fd-e589-4837-8df4-93a479cbafe3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:11.206135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed429fd-e589-4837-8df4-93a479cbafe3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:11.211815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 566/1130] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:15:11 2022] GET /identity/v3/role_assignments?user.id=8298369a0770410086517e9de2031ba8&scope.project.id=c3e0ded9bb254e2888b04c1f0d31898d&role.id=d123411b649d4acb8648d0c0b80aae6d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:13.121446 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49afdfe-0752-4198-8fd5-d30464a840bd None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:13.122019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49afdfe-0752-4198-8fd5-d30464a840bd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:13.122019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49afdfe-0752-4198-8fd5-d30464a840bd None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:13.123357 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 565/1131] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:15:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:15:13.129112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d296e4-6ee0-4b20-8a5b-afadee4b916b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:13.129532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d296e4-6ee0-4b20-8a5b-afadee4b916b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:13.129641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d296e4-6ee0-4b20-8a5b-afadee4b916b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:13.148076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-90d296e4-6ee0-4b20-8a5b-afadee4b916b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:15:13.149238 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-90d296e4-6ee0-4b20-8a5b-afadee4b916b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:15:13.215851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 567/1132] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:15:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:15:13.484322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49afdfe-0752-4198-8fd5-d30464a840bd None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:13.588221 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49afdfe-0752-4198-8fd5-d30464a840bd None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:13.623025 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026cbf7b-ee67-423c-ad00-8acb0cacdf26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:13.623649 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cbf7b-ee67-423c-ad00-8acb0cacdf26 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:13.623922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cbf7b-ee67-423c-ad00-8acb0cacdf26 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:13.624251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026cbf7b-ee67-423c-ad00-8acb0cacdf26 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:13.625536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cbf7b-ee67-423c-ad00-8acb0cacdf26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:13.627154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026cbf7b-ee67-423c-ad00-8acb0cacdf26 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:13.630008 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 566/1133] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:15:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:13.636007 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d296e4-6ee0-4b20-8a5b-afadee4b916b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:13.711571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:13.732902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d296e4-6ee0-4b20-8a5b-afadee4b916b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:13.765933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c848c302-0f6b-4a46-a441-708a3daa94eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:13.766492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848c302-0f6b-4a46-a441-708a3daa94eb None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:13.766768 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848c302-0f6b-4a46-a441-708a3daa94eb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:13.767033 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c848c302-0f6b-4a46-a441-708a3daa94eb None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:13.767655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848c302-0f6b-4a46-a441-708a3daa94eb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:13.769060 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c848c302-0f6b-4a46-a441-708a3daa94eb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:13.795727 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 568/1134] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 15:15:13 2022] POST /identity/v3/projects => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:15:15.823214 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36138fcf-04c4-4d40-a310-12a521a6c831 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:15.823713 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36138fcf-04c4-4d40-a310-12a521a6c831 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:15.823713 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36138fcf-04c4-4d40-a310-12a521a6c831 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:15.825041 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 567/1135] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:15:15.831238 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3b552-66a9-4285-8498-55dd8bdc4af0 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:15.831704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3b552-66a9-4285-8498-55dd8bdc4af0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:15.831798 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d3b552-66a9-4285-8498-55dd8bdc4af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:15.850436 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f9d3b552-66a9-4285-8498-55dd8bdc4af0 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:15:15.851619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f9d3b552-66a9-4285-8498-55dd8bdc4af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:15:15.917183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 569/1136] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:15:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:15:16.193462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36138fcf-04c4-4d40-a310-12a521a6c831 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:16.253652 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:16.295172 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36138fcf-04c4-4d40-a310-12a521a6c831 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:16.328741 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5b74c-8494-42d1-a8e2-f2773e41c0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:16.329361 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5b74c-8494-42d1-a8e2-f2773e41c0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:16.329637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5b74c-8494-42d1-a8e2-f2773e41c0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:16.329903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5b74c-8494-42d1-a8e2-f2773e41c0f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:16.331607 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5b74c-8494-42d1-a8e2-f2773e41c0f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:16.333018 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5b74c-8494-42d1-a8e2-f2773e41c0f5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:16.334619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 568/1137] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:15:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:16.340541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3b552-66a9-4285-8498-55dd8bdc4af0 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:16.398158 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:16.438410 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d3b552-66a9-4285-8498-55dd8bdc4af0 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:16.472086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b29255-2080-4ed3-9008-48cd1d2e4fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:16.472608 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b29255-2080-4ed3-9008-48cd1d2e4fc7 None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:16.472881 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b29255-2080-4ed3-9008-48cd1d2e4fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:16.473145 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b29255-2080-4ed3-9008-48cd1d2e4fc7 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:16.473773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b29255-2080-4ed3-9008-48cd1d2e4fc7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:16.475208 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b29255-2080-4ed3-9008-48cd1d2e4fc7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:16.496607 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 570/1138] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:15:16 2022] POST /identity/v3/users => generated 286 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:15:18.460837 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d6430-b6ef-4e56-840c-5651938dfecf None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:18.461371 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d6430-b6ef-4e56-840c-5651938dfecf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:18.461371 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d6430-b6ef-4e56-840c-5651938dfecf None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:18.462845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 569/1139] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:15:18.468610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9f2c0-46e3-401d-a1e9-95f8af1479f9 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:18.469046 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9f2c0-46e3-401d-a1e9-95f8af1479f9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:18.469255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf9f2c0-46e3-401d-a1e9-95f8af1479f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:18.487226 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-acf9f2c0-46e3-401d-a1e9-95f8af1479f9 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:15:18.488396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-acf9f2c0-46e3-401d-a1e9-95f8af1479f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:15:18.553600 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 571/1140] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:15:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:15:18.824184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d6430-b6ef-4e56-840c-5651938dfecf None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:18.925573 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d6430-b6ef-4e56-840c-5651938dfecf None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:18.951226 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:18.959076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbdcda-4cf3-402e-8f77-35821b01d0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:18.959697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbdcda-4cf3-402e-8f77-35821b01d0bc None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:18.959971 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbdcda-4cf3-402e-8f77-35821b01d0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:18.960238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcbdcda-4cf3-402e-8f77-35821b01d0bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:18.970057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcbdcda-4cf3-402e-8f77-35821b01d0bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:15:18.974765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-7fcbdcda-4cf3-402e-8f77-35821b01d0bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:15:18.974765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:15:18.974765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:15:18.974765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:15:18.974765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:15:18.974765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:15:18.974765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:15:18.974765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:15:18.974765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:15:18.974765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:15:18.974765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:15:18.974765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:15:18.974765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:15:18.974765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:15:18.974765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:15:18.974765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:15:18.975914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:15:18.975914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:15:18.975914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:15:18.975914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:15:18.975914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:15:18.975914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:15:18.975914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:15:18.975914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:15:18.975914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:15:18.975914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:15:18.975914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:15:18.975914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:15:18.975914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:15:18.975914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:15:18.975914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:15:18.975914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:15:18.976889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:15:18.976889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:15:18.976889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:15:18.976889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:15:18.976889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:15:18.976889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:15:18.976889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:15:18.976889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:15:18.976889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:15:18.976889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:15:18.976889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:15:18.976889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:15:18.976889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 570/1141] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:15:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:15:18.981890 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9f2c0-46e3-401d-a1e9-95f8af1479f9 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:19.027859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:19.080402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf9f2c0-46e3-401d-a1e9-95f8af1479f9 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:19.114423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d354e-cb1a-4a31-814c-f7b2f0e07cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:19.115016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d354e-cb1a-4a31-814c-f7b2f0e07cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:19.115291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d354e-cb1a-4a31-814c-f7b2f0e07cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:19.115559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57d354e-cb1a-4a31-814c-f7b2f0e07cd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:19.116212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d354e-cb1a-4a31-814c-f7b2f0e07cd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:15:19.116510 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d354e-cb1a-4a31-814c-f7b2f0e07cd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:19.117912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57d354e-cb1a-4a31-814c-f7b2f0e07cd0 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:19.125717 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 572/1142] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:15:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:19.131568 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbdcda-4cf3-402e-8f77-35821b01d0bc None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:19.229512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcbdcda-4cf3-402e-8f77-35821b01d0bc None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:19.263550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4685b4-524d-4964-b08d-a8f960c55e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:19.264162 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4685b4-524d-4964-b08d-a8f960c55e8e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:19.264441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4685b4-524d-4964-b08d-a8f960c55e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:19.264708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4685b4-524d-4964-b08d-a8f960c55e8e None admin] PATH_INFO: `/v3/users/5e422c63a74943d7b6ae6d335ad182f1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:19.279310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4685b4-524d-4964-b08d-a8f960c55e8e None admin] RBAC: Authorizing `identity:get_user(user_id=5e422c63a74943d7b6ae6d335ad182f1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:19.280750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4685b4-524d-4964-b08d-a8f960c55e8e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:19.282365 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 571/1143] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:15:19 2022] GET /identity/v3/users/5e422c63a74943d7b6ae6d335ad182f1 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:19.288865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d354e-cb1a-4a31-814c-f7b2f0e07cd0 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:19.387415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57d354e-cb1a-4a31-814c-f7b2f0e07cd0 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:19.395914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:19.421226 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e7c49-9d3f-4fd1-b51e-b21a73aa36a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:19.421827 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e7c49-9d3f-4fd1-b51e-b21a73aa36a7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:19.422103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e7c49-9d3f-4fd1-b51e-b21a73aa36a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:19.422403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8e7c49-9d3f-4fd1-b51e-b21a73aa36a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:19.493702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e7c49-9d3f-4fd1-b51e-b21a73aa36a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:19.495175 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8e7c49-9d3f-4fd1-b51e-b21a73aa36a7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:19.593680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 573/1144] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:15:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:15:19.600467 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4685b4-524d-4964-b08d-a8f960c55e8e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:19.646571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:19.701024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4685b4-524d-4964-b08d-a8f960c55e8e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:19.735850 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87280ccf-9fd0-412e-bf61-6d5ecaa2479a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:19.736466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87280ccf-9fd0-412e-bf61-6d5ecaa2479a None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:19.737258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87280ccf-9fd0-412e-bf61-6d5ecaa2479a None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:19.737527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87280ccf-9fd0-412e-bf61-6d5ecaa2479a None admin] PATH_INFO: `/v3/projects/904f0f057f024b3dba42a572233db177` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:19.738675 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87280ccf-9fd0-412e-bf61-6d5ecaa2479a None admin] RBAC: Authorizing `identity:get_project(project_id=904f0f057f024b3dba42a572233db177)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:19.740098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87280ccf-9fd0-412e-bf61-6d5ecaa2479a None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:19.741726 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 572/1145] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:15:19 2022] GET /identity/v3/projects/904f0f057f024b3dba42a572233db177 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:19.747944 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e7c49-9d3f-4fd1-b51e-b21a73aa36a7 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:19.758437 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:19.845743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8e7c49-9d3f-4fd1-b51e-b21a73aa36a7 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:19.879658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121439ce-4d8a-45d0-8a23-5533cf5a52d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:19.880263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121439ce-4d8a-45d0-8a23-5533cf5a52d9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:19.880537 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121439ce-4d8a-45d0-8a23-5533cf5a52d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:19.880803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121439ce-4d8a-45d0-8a23-5533cf5a52d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:19.881519 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121439ce-4d8a-45d0-8a23-5533cf5a52d9 None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=904f0f057f024b3dba42a572233db177, user.id=5e422c63a74943d7b6ae6d335ad182f1) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:15:19.881819 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121439ce-4d8a-45d0-8a23-5533cf5a52d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:19.883277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121439ce-4d8a-45d0-8a23-5533cf5a52d9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:19.888878 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 574/1146] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:15:19 2022] GET /identity/v3/role_assignments?user.id=5e422c63a74943d7b6ae6d335ad182f1&scope.project.id=904f0f057f024b3dba42a572233db177&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:21.753508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6dc9c6-b9d1-4e12-afa6-f38ae9b7e4db None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:21.754083 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6dc9c6-b9d1-4e12-afa6-f38ae9b7e4db None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:21.754083 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6dc9c6-b9d1-4e12-afa6-f38ae9b7e4db None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:21.755384 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 573/1147] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:15:21.760798 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55148c-0708-4e87-891f-3d7d82c87b4e None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:21.761257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55148c-0708-4e87-891f-3d7d82c87b4e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:21.761346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55148c-0708-4e87-891f-3d7d82c87b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:21.779291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ec55148c-0708-4e87-891f-3d7d82c87b4e None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:15:21.780484 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ec55148c-0708-4e87-891f-3d7d82c87b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:15:21.846084 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 575/1148] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:15:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:15:22.113742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6dc9c6-b9d1-4e12-afa6-f38ae9b7e4db None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:22.213084 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6dc9c6-b9d1-4e12-afa6-f38ae9b7e4db None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:22.227893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:22.248187 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b08876-efb9-47e9-9a2d-92c6da40f266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:22.248809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b08876-efb9-47e9-9a2d-92c6da40f266 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:22.249087 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b08876-efb9-47e9-9a2d-92c6da40f266 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:22.249355 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b08876-efb9-47e9-9a2d-92c6da40f266 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:22.259343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b08876-efb9-47e9-9a2d-92c6da40f266 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:15:22.264007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-10b08876-efb9-47e9-9a2d-92c6da40f266 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:15:22.264007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:15:22.264007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:15:22.264007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:15:22.264007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:15:22.264007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:15:22.264007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:15:22.264007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:15:22.264007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:15:22.264007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:15:22.264007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:15:22.264007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:15:22.264007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:15:22.264007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:15:22.264007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:15:22.264007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:15:22.265089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:15:22.265089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:15:22.265089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:15:22.265089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:15:22.265089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:15:22.265089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:15:22.265089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:15:22.265089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:15:22.265089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:15:22.265089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:15:22.265089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:15:22.265089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:15:22.265089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:15:22.265089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:15:22.265089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:15:22.265089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:15:22.265968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:15:22.265968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:15:22.265968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:15:22.265968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:15:22.265968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:15:22.265968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:15:22.265968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:15:22.265968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:15:22.265968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:15:22.265968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:15:22.265968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:15:22.265968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:15:22.265968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 574/1149] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:15:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:15:22.271345 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55148c-0708-4e87-891f-3d7d82c87b4e None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:22.273539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:22.369510 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec55148c-0708-4e87-891f-3d7d82c87b4e None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:22.401809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58bc10-f721-48cd-bdee-1401203c9f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:22.402423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58bc10-f721-48cd-bdee-1401203c9f7c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:22.402698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58bc10-f721-48cd-bdee-1401203c9f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:22.402965 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58bc10-f721-48cd-bdee-1401203c9f7c None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:22.403611 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58bc10-f721-48cd-bdee-1401203c9f7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:15:22.403908 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58bc10-f721-48cd-bdee-1401203c9f7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:22.405259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58bc10-f721-48cd-bdee-1401203c9f7c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:22.413582 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 576/1150] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:15:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:22.419402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b08876-efb9-47e9-9a2d-92c6da40f266 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:22.518790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b08876-efb9-47e9-9a2d-92c6da40f266 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:22.553608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3c3e9-db5d-435c-960d-1dbe6fd6c7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:22.554242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3c3e9-db5d-435c-960d-1dbe6fd6c7af None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:22.554528 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3c3e9-db5d-435c-960d-1dbe6fd6c7af None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:22.554792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb3c3e9-db5d-435c-960d-1dbe6fd6c7af None admin] PATH_INFO: `/v3/users/5e422c63a74943d7b6ae6d335ad182f1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:22.555889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3c3e9-db5d-435c-960d-1dbe6fd6c7af None admin] RBAC: Authorizing `identity:get_user(user_id=5e422c63a74943d7b6ae6d335ad182f1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:22.557302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb3c3e9-db5d-435c-960d-1dbe6fd6c7af None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:22.558893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 575/1151] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:15:22 2022] GET /identity/v3/users/5e422c63a74943d7b6ae6d335ad182f1 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:22.564726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58bc10-f721-48cd-bdee-1401203c9f7c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:22.626526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:22.670045 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58bc10-f721-48cd-bdee-1401203c9f7c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:22.706022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a45f60-4ee6-4258-9bba-3d8fa6969749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:22.706737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a45f60-4ee6-4258-9bba-3d8fa6969749 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:22.707019 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a45f60-4ee6-4258-9bba-3d8fa6969749 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:22.707287 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a45f60-4ee6-4258-9bba-3d8fa6969749 None admin] PATH_INFO: `/v3/projects/904f0f057f024b3dba42a572233db177` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:22.708451 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a45f60-4ee6-4258-9bba-3d8fa6969749 None admin] RBAC: Authorizing `identity:get_project(project_id=904f0f057f024b3dba42a572233db177)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:22.709885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a45f60-4ee6-4258-9bba-3d8fa6969749 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:22.711575 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 577/1152] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:15:22 2022] GET /identity/v3/projects/904f0f057f024b3dba42a572233db177 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:22.717720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3c3e9-db5d-435c-960d-1dbe6fd6c7af None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:22.763597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:22.822956 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb3c3e9-db5d-435c-960d-1dbe6fd6c7af None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:22.857641 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d808dcc3-fb5c-4e2f-a95a-4aa6ec7eedb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:22.858340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808dcc3-fb5c-4e2f-a95a-4aa6ec7eedb2 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:22.858617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808dcc3-fb5c-4e2f-a95a-4aa6ec7eedb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:22.858886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d808dcc3-fb5c-4e2f-a95a-4aa6ec7eedb2 None admin] PATH_INFO: `/v3/projects/904f0f057f024b3dba42a572233db177/users/5e422c63a74943d7b6ae6d335ad182f1/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:22.860798 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808dcc3-fb5c-4e2f-a95a-4aa6ec7eedb2 None admin] RBAC: Authorizing `identity:create_grant(project_id=904f0f057f024b3dba42a572233db177, user_id=5e422c63a74943d7b6ae6d335ad182f1, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:22.862885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d808dcc3-fb5c-4e2f-a95a-4aa6ec7eedb2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:22.875659 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 576/1153] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:15:22 2022] PUT /identity/v3/projects/904f0f057f024b3dba42a572233db177/users/5e422c63a74943d7b6ae6d335ad182f1/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:15:24.745058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cb68a-a6f6-431f-a6a8-4c5422bd6d44 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:24.745584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cb68a-a6f6-431f-a6a8-4c5422bd6d44 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:24.745584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cb68a-a6f6-431f-a6a8-4c5422bd6d44 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:24.746960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 578/1154] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:15:24.752132 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bac727-cc02-42b6-8514-3e6bac067637 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:24.752564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bac727-cc02-42b6-8514-3e6bac067637 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:24.752669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bac727-cc02-42b6-8514-3e6bac067637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:24.770643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d4bac727-cc02-42b6-8514-3e6bac067637 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:15:24.771819 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d4bac727-cc02-42b6-8514-3e6bac067637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:15:24.837371 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 577/1155] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:15:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:15:25.137114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cb68a-a6f6-431f-a6a8-4c5422bd6d44 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:25.235620 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cb68a-a6f6-431f-a6a8-4c5422bd6d44 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:25.263489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:25.269448 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159e0a4-6aaf-4146-ad38-74e5c1af7349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:25.270059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159e0a4-6aaf-4146-ad38-74e5c1af7349 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:25.270366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159e0a4-6aaf-4146-ad38-74e5c1af7349 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:25.270635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c159e0a4-6aaf-4146-ad38-74e5c1af7349 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:25.281182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c159e0a4-6aaf-4146-ad38-74e5c1af7349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:15:25.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-c159e0a4-6aaf-4146-ad38-74e5c1af7349 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:15:25.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:15:25.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:15:25.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:15:25.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:15:25.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:15:25.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:15:25.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:15:25.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:15:25.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:15:25.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:15:25.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:15:25.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:15:25.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:15:25.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:15:25.285797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:15:25.286931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:15:25.286931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:15:25.286931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:15:25.286931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:15:25.286931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:15:25.286931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:15:25.286931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:15:25.286931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:15:25.286931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:15:25.286931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:15:25.286931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:15:25.286931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:15:25.286931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:15:25.286931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:15:25.286931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:15:25.286931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:15:25.287776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:15:25.287776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:15:25.287776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:15:25.287776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:15:25.287776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:15:25.287776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:15:25.287776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:15:25.287776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:15:25.287776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:15:25.287776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:15:25.287776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:15:25.287776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:15:25.287776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 579/1156] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:15:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:15:25.292884 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bac727-cc02-42b6-8514-3e6bac067637 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:25.322938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:25.389968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bac727-cc02-42b6-8514-3e6bac067637 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:25.423043 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9483da-a2ca-4c23-9be1-c51be8107ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:25.423644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9483da-a2ca-4c23-9be1-c51be8107ebf None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:25.423946 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9483da-a2ca-4c23-9be1-c51be8107ebf None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:25.424217 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9483da-a2ca-4c23-9be1-c51be8107ebf None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:25.424875 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9483da-a2ca-4c23-9be1-c51be8107ebf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:15:25.425170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9483da-a2ca-4c23-9be1-c51be8107ebf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:25.426692 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9483da-a2ca-4c23-9be1-c51be8107ebf None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:25.434551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 578/1157] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:15:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:25.442538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159e0a4-6aaf-4146-ad38-74e5c1af7349 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:25.550526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c159e0a4-6aaf-4146-ad38-74e5c1af7349 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:25.585119 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d216c3b-235b-4f04-b9e7-a2851ed8dbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:25.586762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d216c3b-235b-4f04-b9e7-a2851ed8dbfa None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:25.586762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d216c3b-235b-4f04-b9e7-a2851ed8dbfa None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:25.586762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d216c3b-235b-4f04-b9e7-a2851ed8dbfa None admin] PATH_INFO: `/v3/users/5e422c63a74943d7b6ae6d335ad182f1` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:25.587406 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d216c3b-235b-4f04-b9e7-a2851ed8dbfa None admin] RBAC: Authorizing `identity:get_user(user_id=5e422c63a74943d7b6ae6d335ad182f1)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:25.588843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d216c3b-235b-4f04-b9e7-a2851ed8dbfa None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:25.590464 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 580/1158] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:15:25 2022] GET /identity/v3/users/5e422c63a74943d7b6ae6d335ad182f1 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:25.596377 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9483da-a2ca-4c23-9be1-c51be8107ebf None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:25.687120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:25.696645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9483da-a2ca-4c23-9be1-c51be8107ebf None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:25.729954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efcb4a-ccda-4cd9-aecc-6fbe73247250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:25.730579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efcb4a-ccda-4cd9-aecc-6fbe73247250 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:25.730856 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efcb4a-ccda-4cd9-aecc-6fbe73247250 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:25.731124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efcb4a-ccda-4cd9-aecc-6fbe73247250 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:25.798139 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efcb4a-ccda-4cd9-aecc-6fbe73247250 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:25.800551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efcb4a-ccda-4cd9-aecc-6fbe73247250 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:25.890665 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:25.898010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 579/1159] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:15:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:15:25.904325 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d216c3b-235b-4f04-b9e7-a2851ed8dbfa None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:25.965603 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:26.006993 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d216c3b-235b-4f04-b9e7-a2851ed8dbfa None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:26.040000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed94a0-85f8-4c70-b512-0e4d204626a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:26.040618 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed94a0-85f8-4c70-b512-0e4d204626a7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:26.040899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed94a0-85f8-4c70-b512-0e4d204626a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:26.041173 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed94a0-85f8-4c70-b512-0e4d204626a7 None admin] PATH_INFO: `/v3/projects/904f0f057f024b3dba42a572233db177` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:26.042304 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed94a0-85f8-4c70-b512-0e4d204626a7 None admin] RBAC: Authorizing `identity:get_project(project_id=904f0f057f024b3dba42a572233db177)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:26.043739 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ed94a0-85f8-4c70-b512-0e4d204626a7 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:26.045364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 581/1160] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:15:25 2022] GET /identity/v3/projects/904f0f057f024b3dba42a572233db177 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:26.052176 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efcb4a-ccda-4cd9-aecc-6fbe73247250 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:26.148443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efcb4a-ccda-4cd9-aecc-6fbe73247250 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:26.182415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d46ff-a78c-4f6d-bd9c-2989b9e1d9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:26.183006 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d46ff-a78c-4f6d-bd9c-2989b9e1d9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:26.183285 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d46ff-a78c-4f6d-bd9c-2989b9e1d9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:26.183558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d46ff-a78c-4f6d-bd9c-2989b9e1d9f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:26.184276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d46ff-a78c-4f6d-bd9c-2989b9e1d9f2 None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=904f0f057f024b3dba42a572233db177, user.id=5e422c63a74943d7b6ae6d335ad182f1) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:15:26.184585 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d46ff-a78c-4f6d-bd9c-2989b9e1d9f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:26.186008 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d46ff-a78c-4f6d-bd9c-2989b9e1d9f2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:26.191729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 580/1161] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:15:26 2022] GET /identity/v3/role_assignments?user.id=5e422c63a74943d7b6ae6d335ad182f1&scope.project.id=904f0f057f024b3dba42a572233db177&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:28.072129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d743e-3a02-4b85-8acf-834ee3c99971 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:28.072721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d743e-3a02-4b85-8acf-834ee3c99971 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:28.072721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d743e-3a02-4b85-8acf-834ee3c99971 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:28.073978 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 582/1162] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:15:28.079115 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21521111-6c53-4859-8910-a9594ede1fa3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:28.079512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21521111-6c53-4859-8910-a9594ede1fa3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:28.079648 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21521111-6c53-4859-8910-a9594ede1fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:28.098270 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-21521111-6c53-4859-8910-a9594ede1fa3 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:15:28.099448 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-21521111-6c53-4859-8910-a9594ede1fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:15:28.150410 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:28.164406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 581/1163] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:15:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:15:28.433530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d743e-3a02-4b85-8acf-834ee3c99971 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:28.547732 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d743e-3a02-4b85-8acf-834ee3c99971 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:28.574472 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:28.582358 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ddff5-6d20-48b4-94b7-b1aebe36ef55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:28.582962 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ddff5-6d20-48b4-94b7-b1aebe36ef55 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:28.583237 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ddff5-6d20-48b4-94b7-b1aebe36ef55 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:28.583505 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743ddff5-6d20-48b4-94b7-b1aebe36ef55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:28.648697 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ddff5-6d20-48b4-94b7-b1aebe36ef55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:28.650137 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743ddff5-6d20-48b4-94b7-b1aebe36ef55 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:28.750155 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 583/1164] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:15:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:15:28.756888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21521111-6c53-4859-8910-a9594ede1fa3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:28.863106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21521111-6c53-4859-8910-a9594ede1fa3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:28.897594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c735e-62b3-41b0-82fa-7c5931e945ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:28.898240 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c735e-62b3-41b0-82fa-7c5931e945ee None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:28.898527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c735e-62b3-41b0-82fa-7c5931e945ee None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:28.898798 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c735e-62b3-41b0-82fa-7c5931e945ee None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:28.905768 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c735e-62b3-41b0-82fa-7c5931e945ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:28.907250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c735e-62b3-41b0-82fa-7c5931e945ee None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-9c3c735e-62b3-41b0-82fa-7c5931e945ee None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:15:28.919256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:15:28.920939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:15:28.920939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:15:28.920939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:15:28.920939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:15:28.920939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:15:28.920939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:15:28.920939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:15:28.920939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:15:28.920939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:15:28.920939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:15:28.920939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:15:28.920939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:15:28.920939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:15:28.920939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:15:28.920939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:15:28.920939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:15:28.922030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:15:28.922030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:15:28.922030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:15:28.922030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:15:28.922030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:15:28.922030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:15:28.922030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:15:28.922030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:15:28.922030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:15:28.922030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:15:28.922030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 15:15:28.922030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:15:28.922030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 15:15:28.922030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:15:28.922030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 582/1165] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 15:15:28 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:15:28.927116 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ddff5-6d20-48b4-94b7-b1aebe36ef55 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:28.953580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:29.022935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743ddff5-6d20-48b4-94b7-b1aebe36ef55 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:29.056358 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf2d6a-d861-49a9-8df3-8362c8897c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:29.056946 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf2d6a-d861-49a9-8df3-8362c8897c8a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:29.057220 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf2d6a-d861-49a9-8df3-8362c8897c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:29.057485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf2d6a-d861-49a9-8df3-8362c8897c8a None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:29.058132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf2d6a-d861-49a9-8df3-8362c8897c8a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:15:29.058455 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf2d6a-d861-49a9-8df3-8362c8897c8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:29.059803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf2d6a-d861-49a9-8df3-8362c8897c8a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:29.066740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 584/1166] 10.222.0.76 () {56 vars in 1115 bytes} [Wed Aug 24 15:15:28 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:30.933418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6007a-d9af-41a3-b6aa-6da1b07ed766 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:30.933945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6007a-d9af-41a3-b6aa-6da1b07ed766 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:30.933945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f6007a-d9af-41a3-b6aa-6da1b07ed766 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:30.935308 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 583/1167] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:15:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:15:30.940812 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622aa90a-3d8c-4d85-8256-c841dc04d79b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:30.941256 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622aa90a-3d8c-4d85-8256-c841dc04d79b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:30.941366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622aa90a-3d8c-4d85-8256-c841dc04d79b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:30.959390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-622aa90a-3d8c-4d85-8256-c841dc04d79b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:15:30.960573 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-622aa90a-3d8c-4d85-8256-c841dc04d79b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:15:31.029958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 585/1168] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:15:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:15:31.298660 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6007a-d9af-41a3-b6aa-6da1b07ed766 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:31.347480 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:31.401908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f6007a-d9af-41a3-b6aa-6da1b07ed766 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:31.435168 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32653991-1b76-4244-bb55-493ede9abdf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:31.435848 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32653991-1b76-4244-bb55-493ede9abdf6 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:31.436125 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32653991-1b76-4244-bb55-493ede9abdf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:31.436398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32653991-1b76-4244-bb55-493ede9abdf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:31.437059 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32653991-1b76-4244-bb55-493ede9abdf6 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:31.438439 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32653991-1b76-4244-bb55-493ede9abdf6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:31.454454 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 584/1169] 10.222.0.76 () {60 vars in 1138 bytes} [Wed Aug 24 15:15:31 2022] POST /identity/v3/domains => generated 267 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:15:33.360745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454177f-d006-42ee-b8ea-d38d86da110c None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:33.361274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454177f-d006-42ee-b8ea-d38d86da110c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:33.361274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7454177f-d006-42ee-b8ea-d38d86da110c None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:33.362670 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 586/1170] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:15:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:15:33.368077 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25b6e1-5646-4265-9c29-a98646f390ec None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:33.368495 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25b6e1-5646-4265-9c29-a98646f390ec None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:33.368624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d25b6e1-5646-4265-9c29-a98646f390ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:33.387006 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4d25b6e1-5646-4265-9c29-a98646f390ec None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:15:33.388184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4d25b6e1-5646-4265-9c29-a98646f390ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:15:33.454915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 585/1171] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:15:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:15:33.720723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454177f-d006-42ee-b8ea-d38d86da110c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:33.744975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:33.827817 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7454177f-d006-42ee-b8ea-d38d86da110c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:33.860523 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e0386f-ca34-4cf9-80c9-3efa3a0bd52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:33.861137 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0386f-ca34-4cf9-80c9-3efa3a0bd52c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:33.861410 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0386f-ca34-4cf9-80c9-3efa3a0bd52c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:33.861676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e0386f-ca34-4cf9-80c9-3efa3a0bd52c None admin] PATH_INFO: `/v3/domains/d9236a20babf467581b417bf11148297` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:33.872128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0386f-ca34-4cf9-80c9-3efa3a0bd52c None admin] RBAC: Authorizing `identity:get_domain(domain_id=d9236a20babf467581b417bf11148297)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:33.873552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e0386f-ca34-4cf9-80c9-3efa3a0bd52c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:33.875139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 587/1172] 10.222.0.76 () {56 vars in 1183 bytes} [Wed Aug 24 15:15:33 2022] GET /identity/v3/domains/d9236a20babf467581b417bf11148297 => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:33.881575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25b6e1-5646-4265-9c29-a98646f390ec None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:33.920758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:33.980988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d25b6e1-5646-4265-9c29-a98646f390ec None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:34.014541 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b20ff58-2611-42b9-ab09-457fa5cd0eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:34.015141 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20ff58-2611-42b9-ab09-457fa5cd0eff None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:34.015418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20ff58-2611-42b9-ab09-457fa5cd0eff None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:34.015687 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b20ff58-2611-42b9-ab09-457fa5cd0eff None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:34.016329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20ff58-2611-42b9-ab09-457fa5cd0eff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:34.017729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b20ff58-2611-42b9-ab09-457fa5cd0eff None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:34.048307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 586/1173] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 15:15:33 2022] POST /identity/v3/projects => generated 361 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:15:35.981201 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12021b-a2d3-40bb-a452-58575070919f None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:35.981737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12021b-a2d3-40bb-a452-58575070919f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:35.981849 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12021b-a2d3-40bb-a452-58575070919f None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:35.983388 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 588/1174] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:15:35.989197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e827d-dd75-44c2-85f1-fa615bc11e81 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:35.989614 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e827d-dd75-44c2-85f1-fa615bc11e81 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:35.989724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80e827d-dd75-44c2-85f1-fa615bc11e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:36.007832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d80e827d-dd75-44c2-85f1-fa615bc11e81 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:15:36.009005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d80e827d-dd75-44c2-85f1-fa615bc11e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:15:36.073931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 587/1175] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:15:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:15:36.355880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12021b-a2d3-40bb-a452-58575070919f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:36.438620 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:36.457932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12021b-a2d3-40bb-a452-58575070919f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:36.491275 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf521b4-2ce6-4fa7-b18a-3f200d5347ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:36.491896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf521b4-2ce6-4fa7-b18a-3f200d5347ec None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:36.492174 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf521b4-2ce6-4fa7-b18a-3f200d5347ec None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:36.492448 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf521b4-2ce6-4fa7-b18a-3f200d5347ec None admin] PATH_INFO: `/v3/domains/d9236a20babf467581b417bf11148297` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:36.493571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf521b4-2ce6-4fa7-b18a-3f200d5347ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=d9236a20babf467581b417bf11148297)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:36.495027 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf521b4-2ce6-4fa7-b18a-3f200d5347ec None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:36.496601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 589/1176] 10.222.0.76 () {56 vars in 1183 bytes} [Wed Aug 24 15:15:36 2022] GET /identity/v3/domains/d9236a20babf467581b417bf11148297 => generated 267 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:36.502401 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e827d-dd75-44c2-85f1-fa615bc11e81 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:36.523383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:36.601249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80e827d-dd75-44c2-85f1-fa615bc11e81 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:36.633969 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70ba38-ebb7-4d0b-b67d-f218101e1ce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:36.634531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70ba38-ebb7-4d0b-b67d-f218101e1ce5 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:36.634809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70ba38-ebb7-4d0b-b67d-f218101e1ce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:36.635076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70ba38-ebb7-4d0b-b67d-f218101e1ce5 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:36.635705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70ba38-ebb7-4d0b-b67d-f218101e1ce5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:36.637119 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c70ba38-ebb7-4d0b-b67d-f218101e1ce5 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:36.657218 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 588/1177] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:15:36 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:15:38.578398 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a62aaf-028f-4b2c-87bc-eaad0232c743 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:38.578956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a62aaf-028f-4b2c-87bc-eaad0232c743 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:38.578956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a62aaf-028f-4b2c-87bc-eaad0232c743 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:38.580280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 590/1178] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:15:38.586185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ef4d7-b57d-4030-a673-4546560c1d57 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:38.586652 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ef4d7-b57d-4030-a673-4546560c1d57 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:38.586760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172ef4d7-b57d-4030-a673-4546560c1d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:38.604671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-172ef4d7-b57d-4030-a673-4546560c1d57 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:15:38.605952 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-172ef4d7-b57d-4030-a673-4546560c1d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:15:38.671067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 589/1179] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:15:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:15:38.938952 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a62aaf-028f-4b2c-87bc-eaad0232c743 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:39.052532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a62aaf-028f-4b2c-87bc-eaad0232c743 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:39.084863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d265a9b-4041-494b-b1be-401dfc785aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:39.085502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d265a9b-4041-494b-b1be-401dfc785aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:39.085780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d265a9b-4041-494b-b1be-401dfc785aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:39.086050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d265a9b-4041-494b-b1be-401dfc785aa1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:39.096981 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d265a9b-4041-494b-b1be-401dfc785aa1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:15:39.101652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-1d265a9b-4041-494b-b1be-401dfc785aa1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:15:39.101652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:15:39.101652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:15:39.101652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:15:39.101652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:15:39.101652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:15:39.101652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:15:39.101652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:15:39.101652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:15:39.101652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:15:39.101652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:15:39.101652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:15:39.101652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:15:39.101652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:15:39.101652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:15:39.101652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:15:39.102669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:15:39.102669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:15:39.102669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:15:39.102669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:15:39.102669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:15:39.102669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:15:39.102669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:15:39.102669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:15:39.102669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:15:39.102669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:15:39.102669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:15:39.102669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:15:39.102669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:15:39.102669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:15:39.102669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:15:39.102669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:15:39.103531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:15:39.103531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:15:39.103531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:15:39.103531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:15:39.103531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:15:39.103531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:15:39.103531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:15:39.103531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:15:39.103531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:15:39.103531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:15:39.103531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:15:39.103531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:15:39.103531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 591/1180] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:15:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:15:39.108898 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ef4d7-b57d-4030-a673-4546560c1d57 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:39.117643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:39.208037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172ef4d7-b57d-4030-a673-4546560c1d57 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:39.240958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffff95-9c99-45a6-93ac-3aff62dc9a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:39.241552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffff95-9c99-45a6-93ac-3aff62dc9a39 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:39.241827 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffff95-9c99-45a6-93ac-3aff62dc9a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:39.242094 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffff95-9c99-45a6-93ac-3aff62dc9a39 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:39.242775 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffff95-9c99-45a6-93ac-3aff62dc9a39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:15:39.243073 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffff95-9c99-45a6-93ac-3aff62dc9a39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:39.244416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffff95-9c99-45a6-93ac-3aff62dc9a39 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:39.252292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 590/1181] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:15:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:39.258764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d265a9b-4041-494b-b1be-401dfc785aa1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:39.271291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:39.357959 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d265a9b-4041-494b-b1be-401dfc785aa1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:39.390749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa306f-6175-4dc8-89f4-a501df32e6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:39.391367 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa306f-6175-4dc8-89f4-a501df32e6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:39.391646 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa306f-6175-4dc8-89f4-a501df32e6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:39.391919 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa306f-6175-4dc8-89f4-a501df32e6d3 None admin] PATH_INFO: `/v3/users/bfb83f10b4ad44dc99a92d26b190f37f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:39.404800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa306f-6175-4dc8-89f4-a501df32e6d3 None admin] RBAC: Authorizing `identity:get_user(user_id=bfb83f10b4ad44dc99a92d26b190f37f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:39.406255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa306f-6175-4dc8-89f4-a501df32e6d3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:39.407836 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 592/1182] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:15:39 2022] GET /identity/v3/users/bfb83f10b4ad44dc99a92d26b190f37f => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:39.414334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffff95-9c99-45a6-93ac-3aff62dc9a39 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:39.479155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:39.519892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffff95-9c99-45a6-93ac-3aff62dc9a39 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:39.558696 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a8d13-195f-4c00-b4aa-0b51a499c396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:39.559295 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a8d13-195f-4c00-b4aa-0b51a499c396 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:39.559568 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a8d13-195f-4c00-b4aa-0b51a499c396 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:39.559832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9a8d13-195f-4c00-b4aa-0b51a499c396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:39.626976 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a8d13-195f-4c00-b4aa-0b51a499c396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:39.628528 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9a8d13-195f-4c00-b4aa-0b51a499c396 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:39.690571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:39.727241 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 591/1183] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:15:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:15:39.733211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa306f-6175-4dc8-89f4-a501df32e6d3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:39.798199 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:39.831916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa306f-6175-4dc8-89f4-a501df32e6d3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:39.865011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f5e4f4-c8bc-4dd3-b49e-c27179603ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:39.865628 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5e4f4-c8bc-4dd3-b49e-c27179603ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:39.865908 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5e4f4-c8bc-4dd3-b49e-c27179603ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:39.866206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f5e4f4-c8bc-4dd3-b49e-c27179603ea9 None admin] PATH_INFO: `/v3/projects/6c39149b507a41d9af532f62c2813444` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:39.867335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5e4f4-c8bc-4dd3-b49e-c27179603ea9 None admin] RBAC: Authorizing `identity:get_project(project_id=6c39149b507a41d9af532f62c2813444)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:39.868761 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f5e4f4-c8bc-4dd3-b49e-c27179603ea9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:39.870407 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 593/1184] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:15:39 2022] GET /identity/v3/projects/6c39149b507a41d9af532f62c2813444 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:39.876493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a8d13-195f-4c00-b4aa-0b51a499c396 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:39.989817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9a8d13-195f-4c00-b4aa-0b51a499c396 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:40.023312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b774ac-c1c6-4531-bd38-8f0e80de4319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:40.023911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b774ac-c1c6-4531-bd38-8f0e80de4319 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:40.024181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b774ac-c1c6-4531-bd38-8f0e80de4319 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:40.024446 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b774ac-c1c6-4531-bd38-8f0e80de4319 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:40.025155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b774ac-c1c6-4531-bd38-8f0e80de4319 None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=6c39149b507a41d9af532f62c2813444, user.id=bfb83f10b4ad44dc99a92d26b190f37f) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:15:40.025454 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b774ac-c1c6-4531-bd38-8f0e80de4319 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:40.027148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b774ac-c1c6-4531-bd38-8f0e80de4319 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:40.032330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 592/1185] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:15:39 2022] GET /identity/v3/role_assignments?user.id=bfb83f10b4ad44dc99a92d26b190f37f&scope.project.id=6c39149b507a41d9af532f62c2813444&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:41.921330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce71b42-c15e-4782-bf68-947bbd5edf4c None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:41.921859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce71b42-c15e-4782-bf68-947bbd5edf4c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:41.921859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce71b42-c15e-4782-bf68-947bbd5edf4c None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:41.923255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 594/1186] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:15:41.929161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcd66d-40c8-4584-a74b-1b19c8dd96f2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:41.929589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcd66d-40c8-4584-a74b-1b19c8dd96f2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:41.929709 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfdcd66d-40c8-4584-a74b-1b19c8dd96f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:41.947976 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-bfdcd66d-40c8-4584-a74b-1b19c8dd96f2 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:15:41.949178 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bfdcd66d-40c8-4584-a74b-1b19c8dd96f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:15:41.975327 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:42.019341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 593/1187] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:15:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:15:42.287526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce71b42-c15e-4782-bf68-947bbd5edf4c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:42.385692 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce71b42-c15e-4782-bf68-947bbd5edf4c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:42.419314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fc5f3-b23e-4e6b-bcc4-6e67c39cede1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:42.419937 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fc5f3-b23e-4e6b-bcc4-6e67c39cede1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:42.420209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fc5f3-b23e-4e6b-bcc4-6e67c39cede1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:42.420473 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fc5f3-b23e-4e6b-bcc4-6e67c39cede1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:42.423090 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:42.432030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147fc5f3-b23e-4e6b-bcc4-6e67c39cede1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:15:42.436757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-147fc5f3-b23e-4e6b-bcc4-6e67c39cede1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:15:42.436757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:15:42.436757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:15:42.436757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:15:42.436757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:15:42.436757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:15:42.436757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:15:42.436757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:15:42.436757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:15:42.436757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:15:42.436757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:15:42.436757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:15:42.436757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:15:42.436757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:15:42.436757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:15:42.436757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:15:42.437883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:15:42.437883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:15:42.437883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:15:42.437883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:15:42.437883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:15:42.437883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:15:42.437883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:15:42.437883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:15:42.437883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:15:42.437883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:15:42.437883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:15:42.437883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:15:42.437883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:15:42.437883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:15:42.437883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:15:42.437883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:15:42.439117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:15:42.439117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:15:42.439117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:15:42.439117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:15:42.439117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:15:42.439117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:15:42.439117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:15:42.439117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:15:42.439117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:15:42.439117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:15:42.439117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:15:42.439117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:15:42.439117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 595/1188] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:15:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:15:42.443934 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdcd66d-40c8-4584-a74b-1b19c8dd96f2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:42.543639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfdcd66d-40c8-4584-a74b-1b19c8dd96f2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:42.577191 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd7021-21c4-474b-a7f0-1b1fccfac1d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:42.577792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd7021-21c4-474b-a7f0-1b1fccfac1d0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:42.578071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd7021-21c4-474b-a7f0-1b1fccfac1d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:42.578381 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dd7021-21c4-474b-a7f0-1b1fccfac1d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:42.579037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd7021-21c4-474b-a7f0-1b1fccfac1d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:15:42.579337 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd7021-21c4-474b-a7f0-1b1fccfac1d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:42.580682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dd7021-21c4-474b-a7f0-1b1fccfac1d0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:42.588606 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 594/1189] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:15:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:42.594459 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fc5f3-b23e-4e6b-bcc4-6e67c39cede1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:42.692898 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147fc5f3-b23e-4e6b-bcc4-6e67c39cede1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:42.726366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370c8d5-c289-4c0b-a155-b577d6ab9eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:42.726981 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370c8d5-c289-4c0b-a155-b577d6ab9eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:42.727261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370c8d5-c289-4c0b-a155-b577d6ab9eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:42.727533 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8370c8d5-c289-4c0b-a155-b577d6ab9eb8 None admin] PATH_INFO: `/v3/users/bfb83f10b4ad44dc99a92d26b190f37f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:42.728640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370c8d5-c289-4c0b-a155-b577d6ab9eb8 None admin] RBAC: Authorizing `identity:get_user(user_id=bfb83f10b4ad44dc99a92d26b190f37f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:42.730072 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8370c8d5-c289-4c0b-a155-b577d6ab9eb8 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:42.731659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 596/1190] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:15:42 2022] GET /identity/v3/users/bfb83f10b4ad44dc99a92d26b190f37f => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:42.737559 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd7021-21c4-474b-a7f0-1b1fccfac1d0 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:42.762456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:42.837564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dd7021-21c4-474b-a7f0-1b1fccfac1d0 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:42.869971 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508fae9d-ab75-448d-8c10-51be964d61df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:42.870615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fae9d-ab75-448d-8c10-51be964d61df None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:42.870894 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fae9d-ab75-448d-8c10-51be964d61df None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:42.871162 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508fae9d-ab75-448d-8c10-51be964d61df None admin] PATH_INFO: `/v3/projects/6c39149b507a41d9af532f62c2813444` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:42.872270 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508fae9d-ab75-448d-8c10-51be964d61df None admin] RBAC: Authorizing `identity:get_project(project_id=6c39149b507a41d9af532f62c2813444)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:42.873684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508fae9d-ab75-448d-8c10-51be964d61df None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:42.875332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 595/1191] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:15:42 2022] GET /identity/v3/projects/6c39149b507a41d9af532f62c2813444 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:42.881414 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370c8d5-c289-4c0b-a155-b577d6ab9eb8 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:42.942497 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:42.980813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8370c8d5-c289-4c0b-a155-b577d6ab9eb8 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:43.014151 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5467d-f68d-4644-a155-9926b0c3f331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:43.014818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5467d-f68d-4644-a155-9926b0c3f331 None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:43.015098 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5467d-f68d-4644-a155-9926b0c3f331 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:43.015372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5467d-f68d-4644-a155-9926b0c3f331 None admin] PATH_INFO: `/v3/projects/6c39149b507a41d9af532f62c2813444/users/bfb83f10b4ad44dc99a92d26b190f37f/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:43.017263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5467d-f68d-4644-a155-9926b0c3f331 None admin] RBAC: Authorizing `identity:create_grant(project_id=6c39149b507a41d9af532f62c2813444, user_id=bfb83f10b4ad44dc99a92d26b190f37f, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:43.019254 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5467d-f68d-4644-a155-9926b0c3f331 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:43.032342 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 597/1192] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:15:42 2022] PUT /identity/v3/projects/6c39149b507a41d9af532f62c2813444/users/bfb83f10b4ad44dc99a92d26b190f37f/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:15:44.899019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a661dc4-79a5-4236-a246-b58be3989251 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:44.899579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a661dc4-79a5-4236-a246-b58be3989251 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:44.899579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a661dc4-79a5-4236-a246-b58be3989251 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:44.900874 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 596/1193] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:15:44.906501 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670747a1-5e10-4dd1-8ca0-ac600c0d6c4a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:44.906918 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670747a1-5e10-4dd1-8ca0-ac600c0d6c4a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:44.907036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670747a1-5e10-4dd1-8ca0-ac600c0d6c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:44.926446 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-670747a1-5e10-4dd1-8ca0-ac600c0d6c4a None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:15:44.927614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-670747a1-5e10-4dd1-8ca0-ac600c0d6c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:15:44.992724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 598/1194] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:15:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:15:45.259440 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a661dc4-79a5-4236-a246-b58be3989251 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:45.351728 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:45.361426 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a661dc4-79a5-4236-a246-b58be3989251 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:45.395665 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee35ef7-88b8-4e48-94cc-f31ce4a88023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:45.396286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee35ef7-88b8-4e48-94cc-f31ce4a88023 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:45.396562 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee35ef7-88b8-4e48-94cc-f31ce4a88023 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:45.396832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee35ef7-88b8-4e48-94cc-f31ce4a88023 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:45.410597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee35ef7-88b8-4e48-94cc-f31ce4a88023 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:15:45.415324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-cee35ef7-88b8-4e48-94cc-f31ce4a88023 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:15:45.415324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:15:45.415324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:15:45.415324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:15:45.415324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:15:45.415324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:15:45.415324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:15:45.415324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:15:45.415324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:15:45.415324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:15:45.415324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:15:45.415324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:15:45.415324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:15:45.415324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:15:45.415324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:15:45.415324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:15:45.416397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:15:45.416397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:15:45.416397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:15:45.416397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:15:45.416397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:15:45.416397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:15:45.416397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:15:45.416397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:15:45.416397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:15:45.416397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:15:45.416397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:15:45.416397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:15:45.416397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:15:45.416397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:15:45.416397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:15:45.416397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:15:45.417229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:15:45.417229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:15:45.417229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:15:45.417229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:15:45.417229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:15:45.417229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:15:45.417229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:15:45.417229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:15:45.417229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:15:45.417229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:15:45.417229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:15:45.417229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:15:45.417229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 597/1195] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:15:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:15:45.422659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670747a1-5e10-4dd1-8ca0-ac600c0d6c4a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:45.477457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:45.525077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670747a1-5e10-4dd1-8ca0-ac600c0d6c4a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:45.559125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cf32a-f6f2-4969-a60d-f93ba5a888ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:45.559728 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cf32a-f6f2-4969-a60d-f93ba5a888ff None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:45.560006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cf32a-f6f2-4969-a60d-f93ba5a888ff None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:45.560274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cf32a-f6f2-4969-a60d-f93ba5a888ff None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:45.560931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cf32a-f6f2-4969-a60d-f93ba5a888ff None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:15:45.561234 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cf32a-f6f2-4969-a60d-f93ba5a888ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:45.562631 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cf32a-f6f2-4969-a60d-f93ba5a888ff None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:45.571032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 599/1196] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:15:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:45.577468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee35ef7-88b8-4e48-94cc-f31ce4a88023 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:45.693677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee35ef7-88b8-4e48-94cc-f31ce4a88023 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:45.734132 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d3ea0-fb16-410b-a961-0652d9a9ec58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:45.734788 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d3ea0-fb16-410b-a961-0652d9a9ec58 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:45.735068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d3ea0-fb16-410b-a961-0652d9a9ec58 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:45.735339 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d3ea0-fb16-410b-a961-0652d9a9ec58 None admin] PATH_INFO: `/v3/users/bfb83f10b4ad44dc99a92d26b190f37f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:45.736463 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d3ea0-fb16-410b-a961-0652d9a9ec58 None admin] RBAC: Authorizing `identity:get_user(user_id=bfb83f10b4ad44dc99a92d26b190f37f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:45.738449 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d3ea0-fb16-410b-a961-0652d9a9ec58 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:45.740030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 598/1197] 10.222.0.76 () {56 vars in 1177 bytes} [Wed Aug 24 15:15:45 2022] GET /identity/v3/users/bfb83f10b4ad44dc99a92d26b190f37f => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:45.746024 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cf32a-f6f2-4969-a60d-f93ba5a888ff None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:45.846696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cf32a-f6f2-4969-a60d-f93ba5a888ff None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:45.863652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:45.882456 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c29bcb-25de-4c8b-a1ca-7efc66072dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:45.883052 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c29bcb-25de-4c8b-a1ca-7efc66072dae None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:45.883327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c29bcb-25de-4c8b-a1ca-7efc66072dae None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:45.883594 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c29bcb-25de-4c8b-a1ca-7efc66072dae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:45.953695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c29bcb-25de-4c8b-a1ca-7efc66072dae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:45.955171 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c29bcb-25de-4c8b-a1ca-7efc66072dae None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:46.059332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 600/1198] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:15:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:15:46.065340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d3ea0-fb16-410b-a961-0652d9a9ec58 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:46.088168 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:46.165936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d3ea0-fb16-410b-a961-0652d9a9ec58 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:46.198712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad63b6b-a393-4b49-b4e6-c80df44804f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:46.199322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad63b6b-a393-4b49-b4e6-c80df44804f4 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:46.199596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad63b6b-a393-4b49-b4e6-c80df44804f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:46.199861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad63b6b-a393-4b49-b4e6-c80df44804f4 None admin] PATH_INFO: `/v3/projects/6c39149b507a41d9af532f62c2813444` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:46.200957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad63b6b-a393-4b49-b4e6-c80df44804f4 None admin] RBAC: Authorizing `identity:get_project(project_id=6c39149b507a41d9af532f62c2813444)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:46.202397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad63b6b-a393-4b49-b4e6-c80df44804f4 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:46.204015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 599/1199] 10.222.0.76 () {56 vars in 1186 bytes} [Wed Aug 24 15:15:46 2022] GET /identity/v3/projects/6c39149b507a41d9af532f62c2813444 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:46.210182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c29bcb-25de-4c8b-a1ca-7efc66072dae None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:15:46.228380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:46.313657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c29bcb-25de-4c8b-a1ca-7efc66072dae None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:15:46.349830 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75d2f95-5838-44a7-9a28-3edc94c5c35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:15:46.350442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d2f95-5838-44a7-9a28-3edc94c5c35e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:15:46.350718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d2f95-5838-44a7-9a28-3edc94c5c35e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:15:46.350985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75d2f95-5838-44a7-9a28-3edc94c5c35e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:15:46.351701 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75d2f95-5838-44a7-9a28-3edc94c5c35e None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=6c39149b507a41d9af532f62c2813444, user.id=bfb83f10b4ad44dc99a92d26b190f37f) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:15:46.352003 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75d2f95-5838-44a7-9a28-3edc94c5c35e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:15:46.353442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75d2f95-5838-44a7-9a28-3edc94c5c35e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:15:46.358997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 601/1200] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:15:46 2022] GET /identity/v3/role_assignments?user.id=bfb83f10b4ad44dc99a92d26b190f37f&scope.project.id=6c39149b507a41d9af532f62c2813444&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:15:56.194503 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=56 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:15:56.356461 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=60 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:41.860572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080efb4-07b8-4895-a119-9e4609d2a75b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:41.860572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080efb4-07b8-4895-a119-9e4609d2a75b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:41.860572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c080efb4-07b8-4895-a119-9e4609d2a75b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:41.861634 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 600/1201] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:16:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:16:41.868331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99ac9c-7ddb-4520-80a1-c243ee1dfb0f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:41.868331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99ac9c-7ddb-4520-80a1-c243ee1dfb0f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:41.868331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d99ac9c-7ddb-4520-80a1-c243ee1dfb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:41.890494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5d99ac9c-7ddb-4520-80a1-c243ee1dfb0f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:16:41.892298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5d99ac9c-7ddb-4520-80a1-c243ee1dfb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:16:41.968614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 602/1202] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:16:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:16:42.237654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080efb4-07b8-4895-a119-9e4609d2a75b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:42.372761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c080efb4-07b8-4895-a119-9e4609d2a75b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:42.424370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d96e3f-8e11-4a92-86f2-9fe952039103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:42.424370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d96e3f-8e11-4a92-86f2-9fe952039103 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:42.424370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d96e3f-8e11-4a92-86f2-9fe952039103 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:42.424370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d96e3f-8e11-4a92-86f2-9fe952039103 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:42.439940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d96e3f-8e11-4a92-86f2-9fe952039103 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:42.440945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d96e3f-8e11-4a92-86f2-9fe952039103 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:42.457195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-84d96e3f-8e11-4a92-86f2-9fe952039103 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:16:42.457195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:16:42.457195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:16:42.457195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:16:42.457195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:16:42.457195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:16:42.457195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:16:42.457195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:16:42.457195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:16:42.457195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:16:42.457195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:16:42.457195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:16:42.457195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:16:42.457195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:16:42.457195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:16:42.457195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:16:42.458380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:16:42.459345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:16:42.459345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:16:42.459345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:16:42.459345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 601/1203] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:16:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:16:42.465565 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99ac9c-7ddb-4520-80a1-c243ee1dfb0f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:42.596566 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d99ac9c-7ddb-4520-80a1-c243ee1dfb0f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:42.639719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:42.646692 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce325668-f2f6-4793-8450-5580e5091437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:42.647879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce325668-f2f6-4793-8450-5580e5091437 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:42.648504 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce325668-f2f6-4793-8450-5580e5091437 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:42.649045 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce325668-f2f6-4793-8450-5580e5091437 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:42.650146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce325668-f2f6-4793-8450-5580e5091437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:42.650596 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce325668-f2f6-4793-8450-5580e5091437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:42.652674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce325668-f2f6-4793-8450-5580e5091437 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:42.672403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 603/1204] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:16:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:42.675460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d96e3f-8e11-4a92-86f2-9fe952039103 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:42.750326 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:42.817664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d96e3f-8e11-4a92-86f2-9fe952039103 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:42.873386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f0121-a99c-4871-aac5-e30fec332e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:42.874258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f0121-a99c-4871-aac5-e30fec332e49 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:42.874783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f0121-a99c-4871-aac5-e30fec332e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:42.875200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f0121-a99c-4871-aac5-e30fec332e49 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:42.876026 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f0121-a99c-4871-aac5-e30fec332e49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:42.877968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f0121-a99c-4871-aac5-e30fec332e49 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:42.918763 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 602/1205] 10.222.0.76 () {60 vars in 1132 bytes} [Wed Aug 24 15:16:42 2022] POST /identity/v3/users => generated 251 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:16:45.145327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11617f-12f3-4637-92f3-49e0562bf4e8 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:45.145973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11617f-12f3-4637-92f3-49e0562bf4e8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:45.145973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d11617f-12f3-4637-92f3-49e0562bf4e8 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:45.147860 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 604/1206] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:16:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:16:45.154041 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121996fa-c126-4bf4-9a76-8c561985ad8b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:45.154532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121996fa-c126-4bf4-9a76-8c561985ad8b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:45.154649 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121996fa-c126-4bf4-9a76-8c561985ad8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:45.174677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-121996fa-c126-4bf4-9a76-8c561985ad8b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:16:45.176032 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-121996fa-c126-4bf4-9a76-8c561985ad8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:16:45.254559 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 603/1207] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:16:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:16:45.548945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11617f-12f3-4637-92f3-49e0562bf4e8 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:45.671087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d11617f-12f3-4637-92f3-49e0562bf4e8 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:45.717800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29dbb6-f921-47e4-817a-22658df0750c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:45.718593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29dbb6-f921-47e4-817a-22658df0750c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:45.718874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29dbb6-f921-47e4-817a-22658df0750c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:45.719744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29dbb6-f921-47e4-817a-22658df0750c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:45.734302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29dbb6-f921-47e4-817a-22658df0750c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:16:45.739948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-7c29dbb6-f921-47e4-817a-22658df0750c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:16:45.739948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:16:45.739948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:16:45.739948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:16:45.739948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:16:45.739948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:16:45.739948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:16:45.739948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:16:45.739948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:16:45.739948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:16:45.739948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:16:45.739948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:16:45.739948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:16:45.739948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:16:45.739948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:16:45.739948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:16:45.741200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:16:45.741200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:16:45.741200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:16:45.741200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:16:45.741200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:16:45.741200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:16:45.741200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:16:45.741200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:16:45.741200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:16:45.741200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:16:45.741200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:16:45.741200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:16:45.741200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:16:45.741200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:16:45.741200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:16:45.741200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:16:45.742058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:16:45.742058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:16:45.742058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:16:45.742058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:16:45.742058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:16:45.742058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:16:45.742058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:16:45.742058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:16:45.742058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:16:45.742058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:16:45.742058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:16:45.742058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:16:45.742058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 605/1208] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:16:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:16:45.748423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121996fa-c126-4bf4-9a76-8c561985ad8b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:45.800637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:45.877329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121996fa-c126-4bf4-9a76-8c561985ad8b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:45.920540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f367f-0b1f-48bc-afce-4a4c604f01b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:45.921172 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f367f-0b1f-48bc-afce-4a4c604f01b3 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:45.921486 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f367f-0b1f-48bc-afce-4a4c604f01b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:45.921803 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521f367f-0b1f-48bc-afce-4a4c604f01b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:45.922556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f367f-0b1f-48bc-afce-4a4c604f01b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:45.922924 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f367f-0b1f-48bc-afce-4a4c604f01b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:45.924530 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521f367f-0b1f-48bc-afce-4a4c604f01b3 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:45.939295 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 604/1209] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:16:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:45.945431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29dbb6-f921-47e4-817a-22658df0750c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:46.011459 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:46.069357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29dbb6-f921-47e4-817a-22658df0750c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:46.115098 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050c71a-9e33-4886-89d9-ccf6a4737724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:46.115733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050c71a-9e33-4886-89d9-ccf6a4737724 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:46.116224 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050c71a-9e33-4886-89d9-ccf6a4737724 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:46.116652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2050c71a-9e33-4886-89d9-ccf6a4737724 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:46.128774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050c71a-9e33-4886-89d9-ccf6a4737724 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:46.130277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2050c71a-9e33-4886-89d9-ccf6a4737724 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:46.148423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-2050c71a-9e33-4886-89d9-ccf6a4737724 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:16:46.148423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:16:46.148423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:16:46.148423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:16:46.148423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:16:46.148423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:16:46.148423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:16:46.148423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:16:46.148423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:16:46.148423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:16:46.148423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:16:46.148423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:16:46.148423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:16:46.148423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:16:46.148423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:16:46.148423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:16:46.150076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:16:46.151164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:16:46.151164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:16:46.151164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:16:46.151164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 606/1210] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:16:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:16:46.157623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f367f-0b1f-48bc-afce-4a4c604f01b3 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:46.285466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:46.288665 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521f367f-0b1f-48bc-afce-4a4c604f01b3 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:46.329559 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed67bf2-c534-4851-a530-b914ec3dfc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:46.329559 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed67bf2-c534-4851-a530-b914ec3dfc21 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:46.329559 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed67bf2-c534-4851-a530-b914ec3dfc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:46.329904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed67bf2-c534-4851-a530-b914ec3dfc21 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:46.330317 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed67bf2-c534-4851-a530-b914ec3dfc21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:46.330635 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed67bf2-c534-4851-a530-b914ec3dfc21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:46.331984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed67bf2-c534-4851-a530-b914ec3dfc21 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:46.346254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 605/1211] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:16:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:46.350698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050c71a-9e33-4886-89d9-ccf6a4737724 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:46.461803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2050c71a-9e33-4886-89d9-ccf6a4737724 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:46.464408 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:46.505755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaad639-0bb5-42a2-90f8-27999ea7762d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:46.506492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaad639-0bb5-42a2-90f8-27999ea7762d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:46.506778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaad639-0bb5-42a2-90f8-27999ea7762d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:46.507048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaad639-0bb5-42a2-90f8-27999ea7762d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:46.519821 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaad639-0bb5-42a2-90f8-27999ea7762d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:46.521366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aaad639-0bb5-42a2-90f8-27999ea7762d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:46.536347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-4aaad639-0bb5-42a2-90f8-27999ea7762d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:16:46.536347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:16:46.536347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:16:46.536347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:16:46.536347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:16:46.536347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:16:46.536347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:16:46.536347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:16:46.536347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:16:46.536347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:16:46.536347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:16:46.536347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:16:46.536347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:16:46.536347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:16:46.536347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:16:46.536347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:16:46.537951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:16:46.540132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:16:46.540132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:16:46.540132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:16:46.540132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:16:46.540132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:16:46.540132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:16:46.540132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:16:46.540132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:16:46.540132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:16:46.540132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:16:46.540132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 607/1212] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:16:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:16:46.545791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed67bf2-c534-4851-a530-b914ec3dfc21 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:46.672481 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed67bf2-c534-4851-a530-b914ec3dfc21 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:46.720364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4e3c4-e831-4bae-a3c6-2636322eae6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:46.723220 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4e3c4-e831-4bae-a3c6-2636322eae6e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:46.723220 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4e3c4-e831-4bae-a3c6-2636322eae6e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:46.723220 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4e3c4-e831-4bae-a3c6-2636322eae6e None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:46.723220 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4e3c4-e831-4bae-a3c6-2636322eae6e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:46.723220 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4e3c4-e831-4bae-a3c6-2636322eae6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:46.724141 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca4e3c4-e831-4bae-a3c6-2636322eae6e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:46.746986 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 606/1213] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 24 15:16:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:46.751494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaad639-0bb5-42a2-90f8-27999ea7762d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:46.879334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaad639-0bb5-42a2-90f8-27999ea7762d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:46.937396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea51c7c-13b8-4e63-96d2-7c35dd5c5ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:46.938021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea51c7c-13b8-4e63-96d2-7c35dd5c5ff9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:46.938409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea51c7c-13b8-4e63-96d2-7c35dd5c5ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:46.938688 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea51c7c-13b8-4e63-96d2-7c35dd5c5ff9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:46.980742 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:47.033993 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea51c7c-13b8-4e63-96d2-7c35dd5c5ff9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:47.035652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea51c7c-13b8-4e63-96d2-7c35dd5c5ff9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:47.164402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 608/1214] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:16:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:16:47.170521 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4e3c4-e831-4bae-a3c6-2636322eae6e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:47.191166 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:47.285329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4e3c4-e831-4bae-a3c6-2636322eae6e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:47.321539 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6977931-3df0-4d25-bddb-770eea853759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:47.322249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6977931-3df0-4d25-bddb-770eea853759 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:47.324165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6977931-3df0-4d25-bddb-770eea853759 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:47.324165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6977931-3df0-4d25-bddb-770eea853759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:47.335984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6977931-3df0-4d25-bddb-770eea853759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:47.337484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6977931-3df0-4d25-bddb-770eea853759 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:47.359475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-b6977931-3df0-4d25-bddb-770eea853759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:16:47.359475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:16:47.359475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:16:47.359475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:16:47.359475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:16:47.359475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:16:47.359475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:16:47.359475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:16:47.359475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:16:47.359475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:16:47.359475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:16:47.359475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:16:47.359475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:16:47.359475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:16:47.359475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:16:47.359475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:16:47.361302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:16:47.362583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:16:47.362583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:16:47.362583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:16:47.362583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 607/1215] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:16:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:16:47.366848 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea51c7c-13b8-4e63-96d2-7c35dd5c5ff9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:47.431332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:47.475934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea51c7c-13b8-4e63-96d2-7c35dd5c5ff9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:47.513104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31a173-f2d0-425a-aaa8-13ca5d57b9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:47.513986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31a173-f2d0-425a-aaa8-13ca5d57b9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:47.514459 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31a173-f2d0-425a-aaa8-13ca5d57b9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:47.514885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31a173-f2d0-425a-aaa8-13ca5d57b9f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:47.515696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31a173-f2d0-425a-aaa8-13ca5d57b9f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:47.515991 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31a173-f2d0-425a-aaa8-13ca5d57b9f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:47.517376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31a173-f2d0-425a-aaa8-13ca5d57b9f9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:47.529415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 609/1216] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:16:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:47.535487 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6977931-3df0-4d25-bddb-770eea853759 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:47.618665 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:47.653851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6977931-3df0-4d25-bddb-770eea853759 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:47.690809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d1786-083c-4618-9fb1-16fd8ffe18d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:47.691444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d1786-083c-4618-9fb1-16fd8ffe18d8 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:47.691720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d1786-083c-4618-9fb1-16fd8ffe18d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:47.691987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d1786-083c-4618-9fb1-16fd8ffe18d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:47.698787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d1786-083c-4618-9fb1-16fd8ffe18d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:47.700222 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d1786-083c-4618-9fb1-16fd8ffe18d8 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:47.715175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-859d1786-083c-4618-9fb1-16fd8ffe18d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:16:47.715175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:16:47.715175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:16:47.715175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:16:47.715175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:16:47.715175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:16:47.715175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:16:47.715175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:16:47.715175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:16:47.715175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:16:47.715175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:16:47.715175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:16:47.715175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:16:47.715175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:16:47.715175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:16:47.717402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:16:47.718348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:16:47.718348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:16:47.718348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:16:47.718348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:16:47.718348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:16:47.718348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:16:47.718348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:16:47.718348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:16:47.718348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 608/1217] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:16:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:16:47.722402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31a173-f2d0-425a-aaa8-13ca5d57b9f9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:47.844070 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31a173-f2d0-425a-aaa8-13ca5d57b9f9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:47.870663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:47.885180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ead6ab-a3a2-40ab-9a33-9b5afa4aa333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:47.886030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ead6ab-a3a2-40ab-9a33-9b5afa4aa333 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:47.886416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ead6ab-a3a2-40ab-9a33-9b5afa4aa333 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:47.886844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ead6ab-a3a2-40ab-9a33-9b5afa4aa333 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:47.887560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ead6ab-a3a2-40ab-9a33-9b5afa4aa333 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:47.887845 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ead6ab-a3a2-40ab-9a33-9b5afa4aa333 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:47.889306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ead6ab-a3a2-40ab-9a33-9b5afa4aa333 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:47.904747 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 610/1218] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:16:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:47.911730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d1786-083c-4618-9fb1-16fd8ffe18d8 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:48.023666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d1786-083c-4618-9fb1-16fd8ffe18d8 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:48.056260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:48.062908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a8665-aa6e-4885-b420-12fa11083252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:48.063599 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a8665-aa6e-4885-b420-12fa11083252 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:48.063923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a8665-aa6e-4885-b420-12fa11083252 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:48.064369 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a8665-aa6e-4885-b420-12fa11083252 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:48.065121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a8665-aa6e-4885-b420-12fa11083252 None admin] RBAC: Adding query filter params (role.id=59a2b50693a646fc8332b835233486ce, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=fb745b98e3ae4723bad44428646f83e2) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:48.065426 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a8665-aa6e-4885-b420-12fa11083252 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:48.066902 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a8665-aa6e-4885-b420-12fa11083252 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:48.083018 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 609/1219] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:16:47 2022] GET /identity/v3/role_assignments?user.id=fb745b98e3ae4723bad44428646f83e2&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=59a2b50693a646fc8332b835233486ce => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:50.097398 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f896afa-9c91-4fd4-9d9d-cc824f2b2b4b None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:50.097959 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f896afa-9c91-4fd4-9d9d-cc824f2b2b4b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:50.097959 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f896afa-9c91-4fd4-9d9d-cc824f2b2b4b None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:50.099356 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 611/1220] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:16:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:16:50.105007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea79cd-65f5-4b8e-8a93-b41b2659a337 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:50.105560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea79cd-65f5-4b8e-8a93-b41b2659a337 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:50.105560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea79cd-65f5-4b8e-8a93-b41b2659a337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:50.124828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-79ea79cd-65f5-4b8e-8a93-b41b2659a337 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:16:50.126059 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-79ea79cd-65f5-4b8e-8a93-b41b2659a337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:16:50.211697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 610/1221] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:16:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:16:50.476803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f896afa-9c91-4fd4-9d9d-cc824f2b2b4b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:50.590530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f896afa-9c91-4fd4-9d9d-cc824f2b2b4b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:50.629999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a974f07-c706-44f9-8470-34fc58117057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:50.630698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a974f07-c706-44f9-8470-34fc58117057 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:50.631135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a974f07-c706-44f9-8470-34fc58117057 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:50.631431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a974f07-c706-44f9-8470-34fc58117057 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:50.643648 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a974f07-c706-44f9-8470-34fc58117057 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:16:50.648500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-8a974f07-c706-44f9-8470-34fc58117057 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:16:50.648500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:16:50.648500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:16:50.648500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:16:50.648500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:16:50.648500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:16:50.648500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:16:50.648500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:16:50.648500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:16:50.648500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:16:50.648500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:16:50.648500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:16:50.648500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:16:50.648500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:16:50.648500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:16:50.648500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:16:50.649561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:16:50.649561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:16:50.649561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:16:50.649561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:16:50.649561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:16:50.649561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:16:50.649561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:16:50.649561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:16:50.649561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:16:50.649561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:16:50.649561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:16:50.649561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:16:50.649561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:16:50.649561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:16:50.649561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:16:50.649561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:16:50.650444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:16:50.650444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:16:50.650444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:16:50.650444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:16:50.650444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:16:50.650444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:16:50.650444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:16:50.650444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:16:50.650444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:16:50.650444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:16:50.650444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:16:50.650444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:16:50.650444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 612/1222] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:16:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:16:50.657782 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea79cd-65f5-4b8e-8a93-b41b2659a337 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:50.775405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea79cd-65f5-4b8e-8a93-b41b2659a337 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:50.800064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:50.811913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da06fb6-263f-4c15-beab-6d6503c5cbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:50.812512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da06fb6-263f-4c15-beab-6d6503c5cbd1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:50.812801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da06fb6-263f-4c15-beab-6d6503c5cbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:50.813628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da06fb6-263f-4c15-beab-6d6503c5cbd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:50.814336 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da06fb6-263f-4c15-beab-6d6503c5cbd1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:50.814624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da06fb6-263f-4c15-beab-6d6503c5cbd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:50.815980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da06fb6-263f-4c15-beab-6d6503c5cbd1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:50.824320 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 611/1223] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:16:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:50.829988 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a974f07-c706-44f9-8470-34fc58117057 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:50.879562 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:50.944112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a974f07-c706-44f9-8470-34fc58117057 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:50.978599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6bada-38ad-4ba9-9fd9-4cde8d1c8b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:50.979223 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6bada-38ad-4ba9-9fd9-4cde8d1c8b3d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:50.979495 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6bada-38ad-4ba9-9fd9-4cde8d1c8b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:50.979759 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e6bada-38ad-4ba9-9fd9-4cde8d1c8b3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:50.990757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6bada-38ad-4ba9-9fd9-4cde8d1c8b3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:50.992304 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e6bada-38ad-4ba9-9fd9-4cde8d1c8b3d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:51.010179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-60e6bada-38ad-4ba9-9fd9-4cde8d1c8b3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:16:51.010179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:16:51.010179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:16:51.010179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:16:51.010179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:16:51.010179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:16:51.010179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:16:51.010179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:16:51.010179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:16:51.010179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:16:51.010179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:16:51.010179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:16:51.010179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:16:51.010179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:16:51.010179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:16:51.010179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:16:51.011300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:16:51.012402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:16:51.012402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:16:51.012402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:16:51.012402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 613/1224] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:16:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:16:51.018002 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da06fb6-263f-4c15-beab-6d6503c5cbd1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:51.132733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da06fb6-263f-4c15-beab-6d6503c5cbd1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:51.167812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e58d1-563f-4b8f-a545-bba11abc177c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:51.168562 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e58d1-563f-4b8f-a545-bba11abc177c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:51.168859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e58d1-563f-4b8f-a545-bba11abc177c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:51.169131 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262e58d1-563f-4b8f-a545-bba11abc177c None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:51.169767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e58d1-563f-4b8f-a545-bba11abc177c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:51.170064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e58d1-563f-4b8f-a545-bba11abc177c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:51.171441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262e58d1-563f-4b8f-a545-bba11abc177c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:51.185359 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 612/1225] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:51.191652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6bada-38ad-4ba9-9fd9-4cde8d1c8b3d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:51.296397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:51.304373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e6bada-38ad-4ba9-9fd9-4cde8d1c8b3d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:51.342659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff4738-6238-44a3-b867-476418d60c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:51.343355 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff4738-6238-44a3-b867-476418d60c85 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:51.343667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff4738-6238-44a3-b867-476418d60c85 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:51.343973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ff4738-6238-44a3-b867-476418d60c85 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:51.352832 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff4738-6238-44a3-b867-476418d60c85 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:51.354459 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ff4738-6238-44a3-b867-476418d60c85 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:51.368192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-66ff4738-6238-44a3-b867-476418d60c85 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:16:51.368192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:16:51.368192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:16:51.368192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:16:51.368192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:16:51.368192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:16:51.368192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:16:51.368192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:16:51.368192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:16:51.368192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:16:51.368192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:16:51.368192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:16:51.368192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:16:51.368192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:16:51.368192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:16:51.368192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:16:51.369340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:16:51.370659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:16:51.370659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:16:51.370659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:16:51.370659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:16:51.370659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:16:51.370659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:16:51.370659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:16:51.370659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:16:51.370659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:16:51.370659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:16:51.370659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 614/1226] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:16:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:16:51.376423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e58d1-563f-4b8f-a545-bba11abc177c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:51.425981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:51.483289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262e58d1-563f-4b8f-a545-bba11abc177c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:51.523152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5971599-baff-4fe0-9fcc-021a3d6e7da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:51.523765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5971599-baff-4fe0-9fcc-021a3d6e7da2 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:51.524040 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5971599-baff-4fe0-9fcc-021a3d6e7da2 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:51.524302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5971599-baff-4fe0-9fcc-021a3d6e7da2 None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:51.525028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5971599-baff-4fe0-9fcc-021a3d6e7da2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:51.525360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5971599-baff-4fe0-9fcc-021a3d6e7da2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:51.527083 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5971599-baff-4fe0-9fcc-021a3d6e7da2 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:51.546871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 613/1227] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 24 15:16:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:51.553771 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff4738-6238-44a3-b867-476418d60c85 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:51.670734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ff4738-6238-44a3-b867-476418d60c85 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:51.707300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758d0dc-829d-4271-a886-ecad22b02a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:51.708018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758d0dc-829d-4271-a886-ecad22b02a6b None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:51.708327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758d0dc-829d-4271-a886-ecad22b02a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:51.708637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9758d0dc-829d-4271-a886-ecad22b02a6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:51.721572 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758d0dc-829d-4271-a886-ecad22b02a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:51.723309 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9758d0dc-829d-4271-a886-ecad22b02a6b None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:51.738938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-9758d0dc-829d-4271-a886-ecad22b02a6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:16:51.738938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:16:51.738938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:16:51.738938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:16:51.738938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:16:51.738938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:16:51.738938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:16:51.738938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:16:51.738938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:16:51.738938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:16:51.738938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:16:51.738938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:16:51.738938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:16:51.738938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:16:51.738938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:16:51.738938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:16:51.740129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:16:51.741228 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:16:51.741228 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:16:51.741228 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:16:51.741228 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 615/1228] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:16:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:16:51.746786 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5971599-baff-4fe0-9fcc-021a3d6e7da2 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:51.857171 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5971599-baff-4fe0-9fcc-021a3d6e7da2 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:51.859834 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:51.902126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c8f5b-950e-45c5-bc02-b34087150428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:51.902767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c8f5b-950e-45c5-bc02-b34087150428 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:51.903083 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c8f5b-950e-45c5-bc02-b34087150428 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:51.903358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481c8f5b-950e-45c5-bc02-b34087150428 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:51.904007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c8f5b-950e-45c5-bc02-b34087150428 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:51.904311 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c8f5b-950e-45c5-bc02-b34087150428 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:51.905673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481c8f5b-950e-45c5-bc02-b34087150428 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:51.918854 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 614/1229] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:16:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:51.924847 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758d0dc-829d-4271-a886-ecad22b02a6b None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:51.934761 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:52.037193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9758d0dc-829d-4271-a886-ecad22b02a6b None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:52.076916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ac70e-4a51-4ceb-9b41-07efd6f6a03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:52.077590 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ac70e-4a51-4ceb-9b41-07efd6f6a03e None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:52.077897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ac70e-4a51-4ceb-9b41-07efd6f6a03e None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:52.078139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609ac70e-4a51-4ceb-9b41-07efd6f6a03e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:52.086443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ac70e-4a51-4ceb-9b41-07efd6f6a03e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:52.088030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609ac70e-4a51-4ceb-9b41-07efd6f6a03e None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:52.100264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-609ac70e-4a51-4ceb-9b41-07efd6f6a03e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:16:52.100264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:16:52.100264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:16:52.100264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:16:52.100264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:16:52.100264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:16:52.100264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:16:52.100264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:16:52.100264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:16:52.100264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:16:52.100264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:16:52.100264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:16:52.100264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:16:52.100264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:16:52.100264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:16:52.101463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:16:52.102838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:16:52.102838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:16:52.102838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:16:52.102838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:16:52.102838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:16:52.102838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:16:52.102838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:16:52.102838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:16:52.102838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 616/1230] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:16:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:16:52.107297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c8f5b-950e-45c5-bc02-b34087150428 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:52.216723 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481c8f5b-950e-45c5-bc02-b34087150428 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:52.253089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec19118e-d0fb-45bc-8199-6641c44a1583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:52.253725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19118e-d0fb-45bc-8199-6641c44a1583 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:52.253983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19118e-d0fb-45bc-8199-6641c44a1583 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:52.254388 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec19118e-d0fb-45bc-8199-6641c44a1583 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:52.255098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19118e-d0fb-45bc-8199-6641c44a1583 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:52.255444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19118e-d0fb-45bc-8199-6641c44a1583 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:52.256867 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec19118e-d0fb-45bc-8199-6641c44a1583 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:52.269837 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 615/1231] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:16:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:16:52.277278 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ac70e-4a51-4ceb-9b41-07efd6f6a03e None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:52.351262 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:52.392334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609ac70e-4a51-4ceb-9b41-07efd6f6a03e None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:52.433561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6458140-765a-40dd-ab9d-de3575bfd6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:52.434682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6458140-765a-40dd-ab9d-de3575bfd6e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:52.434960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6458140-765a-40dd-ab9d-de3575bfd6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:52.435234 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6458140-765a-40dd-ab9d-de3575bfd6e2 None admin] PATH_INFO: `/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/fb745b98e3ae4723bad44428646f83e2/roles/59a2b50693a646fc8332b835233486ce` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:52.452852 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6458140-765a-40dd-ab9d-de3575bfd6e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c010096b77f4cd3b2f32086b1ba7ea7, user_id=fb745b98e3ae4723bad44428646f83e2, role_id=59a2b50693a646fc8332b835233486ce)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:52.454670 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6458140-765a-40dd-ab9d-de3575bfd6e2 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:52.468618 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 617/1232] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:16:52 2022] PUT /identity/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/fb745b98e3ae4723bad44428646f83e2/roles/59a2b50693a646fc8332b835233486ce => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:16:54.695655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b9080-6a33-47ce-8e15-24944658c246 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:54.696185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b9080-6a33-47ce-8e15-24944658c246 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:54.696185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b9080-6a33-47ce-8e15-24944658c246 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:54.697536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 616/1233] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:16:54.702767 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f4e86-7f7e-4c16-a7a3-d9a0d8681847 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:54.703193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f4e86-7f7e-4c16-a7a3-d9a0d8681847 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:54.703340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f4e86-7f7e-4c16-a7a3-d9a0d8681847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:54.721920 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2f2f4e86-7f7e-4c16-a7a3-d9a0d8681847 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:16:54.723150 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2f2f4e86-7f7e-4c16-a7a3-d9a0d8681847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:16:54.793823 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 618/1234] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:16:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:16:55.087669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b9080-6a33-47ce-8e15-24944658c246 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:55.118427 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:55.203115 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b9080-6a33-47ce-8e15-24944658c246 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:55.244798 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ff4bb-689d-4bde-ad05-fd6b1cad2c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:55.245479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ff4bb-689d-4bde-ad05-fd6b1cad2c63 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:55.245873 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ff4bb-689d-4bde-ad05-fd6b1cad2c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:55.246236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6ff4bb-689d-4bde-ad05-fd6b1cad2c63 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:55.257883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6ff4bb-689d-4bde-ad05-fd6b1cad2c63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:16:55.262683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-6a6ff4bb-689d-4bde-ad05-fd6b1cad2c63 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:16:55.262683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:16:55.262683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:16:55.262683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:16:55.262683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:16:55.262683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:16:55.262683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:16:55.262683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:16:55.262683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:16:55.262683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:16:55.262683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:16:55.262683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:16:55.262683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:16:55.262683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:16:55.262683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:16:55.262683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:16:55.263795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:16:55.263795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:16:55.263795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:16:55.263795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:16:55.263795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:16:55.263795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:16:55.263795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:16:55.263795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:16:55.263795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:16:55.263795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:16:55.263795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:16:55.263795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:16:55.263795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:16:55.263795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:16:55.263795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:16:55.263795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:16:55.264717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:16:55.264717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:16:55.264717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:16:55.264717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:16:55.264717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:16:55.264717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:16:55.264717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:16:55.264717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:16:55.264717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:16:55.264717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:16:55.264717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 15:16:55.264717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:16:55.264717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 617/1235] 10.222.0.76 () {56 vars in 1102 bytes} [Wed Aug 24 15:16:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:16:55.270958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f4e86-7f7e-4c16-a7a3-d9a0d8681847 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:55.320925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:55.383411 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f4e86-7f7e-4c16-a7a3-d9a0d8681847 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:55.436890 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8826ba-902d-4152-9583-2f4d9ccdde62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:55.437496 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8826ba-902d-4152-9583-2f4d9ccdde62 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:55.438442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8826ba-902d-4152-9583-2f4d9ccdde62 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:55.438763 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8826ba-902d-4152-9583-2f4d9ccdde62 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:55.439477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8826ba-902d-4152-9583-2f4d9ccdde62 None admin] RBAC: Adding query filter params (name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:55.439819 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8826ba-902d-4152-9583-2f4d9ccdde62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:55.441398 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8826ba-902d-4152-9583-2f4d9ccdde62 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:55.450311 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 619/1236] 10.222.0.76 () {56 vars in 1103 bytes} [Wed Aug 24 15:16:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:55.457264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ff4bb-689d-4bde-ad05-fd6b1cad2c63 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:55.558366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:55.576471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6ff4bb-689d-4bde-ad05-fd6b1cad2c63 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:55.615697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e77810-82a6-4206-81b9-f5a6635d674c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:55.616362 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e77810-82a6-4206-81b9-f5a6635d674c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:55.616658 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e77810-82a6-4206-81b9-f5a6635d674c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:55.616883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e77810-82a6-4206-81b9-f5a6635d674c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:55.628107 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e77810-82a6-4206-81b9-f5a6635d674c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:55.629560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e77810-82a6-4206-81b9-f5a6635d674c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:55.645159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-79e77810-82a6-4206-81b9-f5a6635d674c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:16:55.645159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:16:55.645159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:16:55.645159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:16:55.645159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:16:55.645159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:16:55.645159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:16:55.645159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:16:55.645159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:16:55.645159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:16:55.645159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:16:55.645159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:16:55.645159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:16:55.645159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:16:55.645159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:16:55.645159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:16:55.646700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:16:55.647597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:16:55.647597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:16:55.647597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:16:55.647597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 618/1237] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:16:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:16:55.652699 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8826ba-902d-4152-9583-2f4d9ccdde62 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:55.760932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8826ba-902d-4152-9583-2f4d9ccdde62 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:55.770830 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:55.799880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc529b-0f55-4124-81cf-638797019f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:55.800557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc529b-0f55-4124-81cf-638797019f5c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:55.800837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc529b-0f55-4124-81cf-638797019f5c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:55.801126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc529b-0f55-4124-81cf-638797019f5c None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:55.801767 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc529b-0f55-4124-81cf-638797019f5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:55.802067 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc529b-0f55-4124-81cf-638797019f5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:55.803523 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc529b-0f55-4124-81cf-638797019f5c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:55.816777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 620/1238] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:16:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:55.822767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e77810-82a6-4206-81b9-f5a6635d674c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:55.936253 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e77810-82a6-4206-81b9-f5a6635d674c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:55.973161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7551c5-fb6e-4ba8-81f0-e17e93d2694b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:55.973798 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7551c5-fb6e-4ba8-81f0-e17e93d2694b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:55.974078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7551c5-fb6e-4ba8-81f0-e17e93d2694b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:55.974383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7551c5-fb6e-4ba8-81f0-e17e93d2694b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:55.977353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:55.982887 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7551c5-fb6e-4ba8-81f0-e17e93d2694b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:55.985037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7551c5-fb6e-4ba8-81f0-e17e93d2694b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:55.998650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0d7551c5-fb6e-4ba8-81f0-e17e93d2694b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:16:55.998650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:16:55.998650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:16:55.998650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:16:55.998650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:16:55.998650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:16:55.998650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:16:55.998650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:16:55.998650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:16:55.998650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:16:55.998650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:16:55.998650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:16:55.998650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:16:55.998650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:16:55.998650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:16:55.998650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:16:55.999674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:16:56.000663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:16:56.000663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:16:56.000663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:16:56.000663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:16:56.000663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:16:56.000663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:16:56.000663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:16:56.000663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:16:56.000663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:16:56.000663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:16:56.000663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 619/1239] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:16:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:16:56.006001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc529b-0f55-4124-81cf-638797019f5c None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:56.116617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc529b-0f55-4124-81cf-638797019f5c None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:56.155469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31166f6-8977-4207-b660-578dc570155a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:56.156106 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31166f6-8977-4207-b660-578dc570155a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:56.156425 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31166f6-8977-4207-b660-578dc570155a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:56.156702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31166f6-8977-4207-b660-578dc570155a None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:56.157437 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31166f6-8977-4207-b660-578dc570155a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:56.157748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31166f6-8977-4207-b660-578dc570155a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:56.159219 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31166f6-8977-4207-b660-578dc570155a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:56.178910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 621/1240] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 24 15:16:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:56.184702 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7551c5-fb6e-4ba8-81f0-e17e93d2694b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:56.291882 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7551c5-fb6e-4ba8-81f0-e17e93d2694b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:56.333502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8105a-4ff4-4821-a50a-9a73731ca3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:56.334115 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8105a-4ff4-4821-a50a-9a73731ca3db None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:56.334417 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8105a-4ff4-4821-a50a-9a73731ca3db None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:56.334687 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8105a-4ff4-4821-a50a-9a73731ca3db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:56.404249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:56.411242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8105a-4ff4-4821-a50a-9a73731ca3db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:56.412708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8105a-4ff4-4821-a50a-9a73731ca3db None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:56.525085 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 620/1241] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:16:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:16:56.531204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31166f6-8977-4207-b660-578dc570155a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:56.559461 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:56.642276 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31166f6-8977-4207-b660-578dc570155a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:56.685195 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bbe2fd-eaee-45ce-9036-d53a59278715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:56.685994 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbe2fd-eaee-45ce-9036-d53a59278715 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:56.686371 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbe2fd-eaee-45ce-9036-d53a59278715 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:56.686670 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbe2fd-eaee-45ce-9036-d53a59278715 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:56.698306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bbe2fd-eaee-45ce-9036-d53a59278715 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:56.699849 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bbe2fd-eaee-45ce-9036-d53a59278715 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:56.717036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-52bbe2fd-eaee-45ce-9036-d53a59278715 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:16:56.717036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:16:56.717036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:16:56.717036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:16:56.717036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:16:56.717036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:16:56.717036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:16:56.717036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:16:56.717036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:16:56.717036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:16:56.717036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:16:56.717036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:16:56.717036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:16:56.717036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:16:56.717036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:16:56.717036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:16:56.718172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:16:56.719185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:16:56.719185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:16:56.719185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:16:56.719185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 622/1242] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:16:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:16:56.724234 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8105a-4ff4-4821-a50a-9a73731ca3db None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:56.825565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:56.831768 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8105a-4ff4-4821-a50a-9a73731ca3db None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:56.870981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e1979-fc4c-45d4-9455-fbc9093e7431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:56.871636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e1979-fc4c-45d4-9455-fbc9093e7431 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:56.871873 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e1979-fc4c-45d4-9455-fbc9093e7431 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:56.872185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e1979-fc4c-45d4-9455-fbc9093e7431 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:56.872944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e1979-fc4c-45d4-9455-fbc9093e7431 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:56.873295 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e1979-fc4c-45d4-9455-fbc9093e7431 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:56.874793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e1979-fc4c-45d4-9455-fbc9093e7431 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:56.888897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 621/1243] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:56.894795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bbe2fd-eaee-45ce-9036-d53a59278715 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:56.990935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:57.012612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bbe2fd-eaee-45ce-9036-d53a59278715 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:57.053275 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a95591-8ec0-434f-afec-0da6c7aef1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:57.053899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a95591-8ec0-434f-afec-0da6c7aef1c1 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:57.054180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a95591-8ec0-434f-afec-0da6c7aef1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:57.054482 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a95591-8ec0-434f-afec-0da6c7aef1c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:57.061194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a95591-8ec0-434f-afec-0da6c7aef1c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:57.062690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a95591-8ec0-434f-afec-0da6c7aef1c1 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:57.073931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-57a95591-8ec0-434f-afec-0da6c7aef1c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:16:57.073931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:16:57.073931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:16:57.073931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:16:57.073931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:16:57.073931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:16:57.073931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:16:57.073931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:16:57.073931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:16:57.073931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:16:57.073931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:16:57.073931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:16:57.073931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:16:57.073931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:16:57.073931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:16:57.075071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:16:57.076485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:16:57.076485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:16:57.076485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:16:57.076485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:16:57.076485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:16:57.076485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:16:57.076485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:16:57.076485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:16:57.076485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 623/1244] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:16:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:16:57.081309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e1979-fc4c-45d4-9455-fbc9093e7431 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:57.195081 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e1979-fc4c-45d4-9455-fbc9093e7431 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:57.236523 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e984691a-f525-44b1-a074-ba8456f16876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:57.237486 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984691a-f525-44b1-a074-ba8456f16876 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:57.237960 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984691a-f525-44b1-a074-ba8456f16876 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:57.238455 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e984691a-f525-44b1-a074-ba8456f16876 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:57.239733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984691a-f525-44b1-a074-ba8456f16876 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:57.240224 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984691a-f525-44b1-a074-ba8456f16876 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:57.242802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e984691a-f525-44b1-a074-ba8456f16876 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:57.261410 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 622/1245] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:16:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:57.268707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a95591-8ec0-434f-afec-0da6c7aef1c1 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:57.379419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a95591-8ec0-434f-afec-0da6c7aef1c1 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:57.420730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d502d1b8-e290-4960-8b2b-012880f93d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:57.421692 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502d1b8-e290-4960-8b2b-012880f93d7f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:57.422409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502d1b8-e290-4960-8b2b-012880f93d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:57.422764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d502d1b8-e290-4960-8b2b-012880f93d7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:57.423618 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502d1b8-e290-4960-8b2b-012880f93d7f None admin] RBAC: Adding query filter params (role.id=59a2b50693a646fc8332b835233486ce, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=fb745b98e3ae4723bad44428646f83e2) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:16:57.424165 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502d1b8-e290-4960-8b2b-012880f93d7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:16:57.425906 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d502d1b8-e290-4960-8b2b-012880f93d7f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:16:57.434484 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 624/1246] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:16:57 2022] GET /identity/v3/role_assignments?user.id=fb745b98e3ae4723bad44428646f83e2&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=59a2b50693a646fc8332b835233486ce => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:16:59.450722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf8e88-d232-490a-9441-cde6a58104d0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:59.451261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf8e88-d232-490a-9441-cde6a58104d0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:59.451261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faf8e88-d232-490a-9441-cde6a58104d0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:59.452600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 623/1247] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:16:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:16:59.458931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d9692-4c0e-45f1-8d29-805f17bda723 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:59.459380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d9692-4c0e-45f1-8d29-805f17bda723 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:59.459499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d9692-4c0e-45f1-8d29-805f17bda723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:16:59.466267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:59.479631 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6f2d9692-4c0e-45f1-8d29-805f17bda723 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:16:59.480810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6f2d9692-4c0e-45f1-8d29-805f17bda723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:16:59.555667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 625/1248] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:16:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:16:59.828500 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf8e88-d232-490a-9441-cde6a58104d0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:16:59.850614 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:16:59.954653 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faf8e88-d232-490a-9441-cde6a58104d0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:16:59.992211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be672e-47b8-4b5a-850d-d4e6977122ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:16:59.992868 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be672e-47b8-4b5a-850d-d4e6977122ed None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:16:59.993172 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be672e-47b8-4b5a-850d-d4e6977122ed None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:16:59.993436 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be672e-47b8-4b5a-850d-d4e6977122ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:00.004690 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be672e-47b8-4b5a-850d-d4e6977122ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:17:00.009444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-89be672e-47b8-4b5a-850d-d4e6977122ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:17:00.009444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:00.009444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:00.009444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:00.009444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:00.009444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:00.009444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:00.009444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:00.009444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:00.009444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:00.009444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:00.009444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:00.009444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:17:00.009444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:17:00.009444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:17:00.009444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:17:00.010725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:17:00.010725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:17:00.010725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:17:00.010725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:17:00.010725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:00.010725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:00.010725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:00.010725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:00.010725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:00.010725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:00.010725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:00.010725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:00.010725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:00.010725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:00.010725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:00.010725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:00.011787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:00.011787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:00.011787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:00.011787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:00.011787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:17:00.011787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:17:00.011787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:17:00.011787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:17:00.011787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:17:00.011787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:17:00.011787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:17:00.011787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:17:00.011787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 624/1249] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:16:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:17:00.017120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d9692-4c0e-45f1-8d29-805f17bda723 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:00.143704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d9692-4c0e-45f1-8d29-805f17bda723 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:00.177334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:00.184850 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2026768-bc78-4db3-8070-dba55cf38db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:00.185553 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2026768-bc78-4db3-8070-dba55cf38db9 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:00.185891 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2026768-bc78-4db3-8070-dba55cf38db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:00.186149 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2026768-bc78-4db3-8070-dba55cf38db9 None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:00.186853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2026768-bc78-4db3-8070-dba55cf38db9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:00.187204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2026768-bc78-4db3-8070-dba55cf38db9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:00.188912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2026768-bc78-4db3-8070-dba55cf38db9 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:00.201088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 626/1250] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:17:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:00.207420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be672e-47b8-4b5a-850d-d4e6977122ed None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:00.302382 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:00.328025 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be672e-47b8-4b5a-850d-d4e6977122ed None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:00.373305 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e3d75-0ba8-4314-9fd9-4f0d49a5a50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:00.373931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e3d75-0ba8-4314-9fd9-4f0d49a5a50e None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:00.374208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e3d75-0ba8-4314-9fd9-4f0d49a5a50e None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:00.374529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e3d75-0ba8-4314-9fd9-4f0d49a5a50e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:00.386756 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e3d75-0ba8-4314-9fd9-4f0d49a5a50e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:00.388510 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6e3d75-0ba8-4314-9fd9-4f0d49a5a50e None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:00.404627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-3a6e3d75-0ba8-4314-9fd9-4f0d49a5a50e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:17:00.404627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:00.404627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:00.404627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:00.404627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:00.404627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:00.404627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:00.404627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:00.404627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:00.404627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:00.404627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:00.404627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:00.404627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:17:00.404627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:17:00.404627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:17:00.404627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:00.405981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:17:00.407747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:17:00.407747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:17:00.407747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:17:00.407747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 625/1251] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:17:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:17:00.412543 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2026768-bc78-4db3-8070-dba55cf38db9 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:00.538132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2026768-bc78-4db3-8070-dba55cf38db9 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:00.580427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0cf7d-5fa5-435a-9112-43855453dea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:00.581040 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0cf7d-5fa5-435a-9112-43855453dea5 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:00.581345 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0cf7d-5fa5-435a-9112-43855453dea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:00.581695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0cf7d-5fa5-435a-9112-43855453dea5 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:00.582642 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0cf7d-5fa5-435a-9112-43855453dea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:00.583036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0cf7d-5fa5-435a-9112-43855453dea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:00.585091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b0cf7d-5fa5-435a-9112-43855453dea5 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:00.599661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 627/1252] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:00.605510 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e3d75-0ba8-4314-9fd9-4f0d49a5a50e None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:00.714021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e3d75-0ba8-4314-9fd9-4f0d49a5a50e None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:00.742104 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:00.751180 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d7812-b13a-4c92-8fba-7c2ae2d28322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:00.751865 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d7812-b13a-4c92-8fba-7c2ae2d28322 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:00.752143 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d7812-b13a-4c92-8fba-7c2ae2d28322 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:00.752408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d7812-b13a-4c92-8fba-7c2ae2d28322 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:00.760860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d7812-b13a-4c92-8fba-7c2ae2d28322 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:00.762313 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d7812-b13a-4c92-8fba-7c2ae2d28322 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:00.773747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a07d7812-b13a-4c92-8fba-7c2ae2d28322 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:17:00.773747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:00.773747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:00.773747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:00.773747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:00.773747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:00.773747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:00.773747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:00.773747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:00.773747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:00.773747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:00.773747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:00.773747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:17:00.773747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:17:00.773747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:17:00.773747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:00.775152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:00.776145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:00.776145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:00.776145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:17:00.776145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:17:00.776145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:17:00.776145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:17:00.776145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:17:00.776145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:17:00.776145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:17:00.776145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:17:00.776145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 626/1253] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:17:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:17:00.782932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0cf7d-5fa5-435a-9112-43855453dea5 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:00.847918 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:00.899964 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b0cf7d-5fa5-435a-9112-43855453dea5 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:00.940122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced625d-3e16-4a98-8394-256708bccc7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:00.941034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced625d-3e16-4a98-8394-256708bccc7f None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:00.941304 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced625d-3e16-4a98-8394-256708bccc7f None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:00.941569 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ced625d-3e16-4a98-8394-256708bccc7f None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:00.942361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced625d-3e16-4a98-8394-256708bccc7f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:00.942672 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced625d-3e16-4a98-8394-256708bccc7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:00.944107 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ced625d-3e16-4a98-8394-256708bccc7f None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:00.963893 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 628/1254] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 24 15:17:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:00.970956 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d7812-b13a-4c92-8fba-7c2ae2d28322 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:01.078016 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d7812-b13a-4c92-8fba-7c2ae2d28322 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:01.114015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e656d-5679-4f8d-bd2b-7f6c6db63d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:01.114655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e656d-5679-4f8d-bd2b-7f6c6db63d1d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:01.114943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e656d-5679-4f8d-bd2b-7f6c6db63d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:01.115193 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786e656d-5679-4f8d-bd2b-7f6c6db63d1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:01.178662 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:01.192245 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e656d-5679-4f8d-bd2b-7f6c6db63d1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:01.193712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786e656d-5679-4f8d-bd2b-7f6c6db63d1d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:01.306350 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 627/1255] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:17:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:17:01.308540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced625d-3e16-4a98-8394-256708bccc7f None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:01.422046 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ced625d-3e16-4a98-8394-256708bccc7f None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:01.440017 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:01.465532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62376fc-0b5c-4202-8914-a8bd2f194355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:01.466248 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62376fc-0b5c-4202-8914-a8bd2f194355 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:01.466528 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62376fc-0b5c-4202-8914-a8bd2f194355 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:01.466800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62376fc-0b5c-4202-8914-a8bd2f194355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:01.478923 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62376fc-0b5c-4202-8914-a8bd2f194355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:01.480678 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62376fc-0b5c-4202-8914-a8bd2f194355 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:01.496662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-f62376fc-0b5c-4202-8914-a8bd2f194355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:17:01.496662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:01.496662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:01.496662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:01.496662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:01.496662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:01.496662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:01.496662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:01.496662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:01.496662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:01.496662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:01.496662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:01.496662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:17:01.496662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:17:01.496662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:17:01.496662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:01.497834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:17:01.498957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:17:01.498957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:17:01.498957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:17:01.498957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 629/1256] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:17:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:17:01.505280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e656d-5679-4f8d-bd2b-7f6c6db63d1d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:01.606095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:01.619308 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786e656d-5679-4f8d-bd2b-7f6c6db63d1d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:01.655842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416d8ed-2c66-42b3-a74b-cb5209debe28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:01.656940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416d8ed-2c66-42b3-a74b-cb5209debe28 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:01.657209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416d8ed-2c66-42b3-a74b-cb5209debe28 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:01.657513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416d8ed-2c66-42b3-a74b-cb5209debe28 None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:01.658148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416d8ed-2c66-42b3-a74b-cb5209debe28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:01.658565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416d8ed-2c66-42b3-a74b-cb5209debe28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:01.659996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416d8ed-2c66-42b3-a74b-cb5209debe28 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:01.678293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 628/1257] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:01.680865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62376fc-0b5c-4202-8914-a8bd2f194355 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:01.800848 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62376fc-0b5c-4202-8914-a8bd2f194355 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:01.834829 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef012eb-0fd0-4210-86c9-d542ff88eaad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:01.835597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef012eb-0fd0-4210-86c9-d542ff88eaad None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:01.835991 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef012eb-0fd0-4210-86c9-d542ff88eaad None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:01.836365 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef012eb-0fd0-4210-86c9-d542ff88eaad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:01.843126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef012eb-0fd0-4210-86c9-d542ff88eaad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:01.844734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef012eb-0fd0-4210-86c9-d542ff88eaad None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:01.856916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-7ef012eb-0fd0-4210-86c9-d542ff88eaad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:17:01.856916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:01.856916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:01.856916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:01.856916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:01.856916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:01.856916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:01.856916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:01.856916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:01.856916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:01.856916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:01.856916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:01.856916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:17:01.856916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:17:01.856916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:01.857980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:01.859077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:17:01.859077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:17:01.859077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:17:01.859077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:17:01.859077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:17:01.859077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:17:01.859077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:17:01.859077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:17:01.859077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 630/1258] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:17:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 15:17:01.864847 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416d8ed-2c66-42b3-a74b-cb5209debe28 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:01.998620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416d8ed-2c66-42b3-a74b-cb5209debe28 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:02.036049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc4857-04d8-4f6e-87d4-f38273a29650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:02.036727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc4857-04d8-4f6e-87d4-f38273a29650 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:02.037010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc4857-04d8-4f6e-87d4-f38273a29650 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:02.037303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc4857-04d8-4f6e-87d4-f38273a29650 None admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:02.038039 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc4857-04d8-4f6e-87d4-f38273a29650 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:02.038414 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc4857-04d8-4f6e-87d4-f38273a29650 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:02.039869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc4857-04d8-4f6e-87d4-f38273a29650 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:02.053270 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 629/1259] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:17:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:02.059368 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef012eb-0fd0-4210-86c9-d542ff88eaad None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:02.092625 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:02.183709 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef012eb-0fd0-4210-86c9-d542ff88eaad None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:02.224473 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2808bb03-04f7-405b-8cc0-be8edfe1bd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:02.225228 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808bb03-04f7-405b-8cc0-be8edfe1bd45 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:02.225495 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808bb03-04f7-405b-8cc0-be8edfe1bd45 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:02.225826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2808bb03-04f7-405b-8cc0-be8edfe1bd45 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:02.226687 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2808bb03-04f7-405b-8cc0-be8edfe1bd45 None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=fb745b98e3ae4723bad44428646f83e2) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:02.227081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2808bb03-04f7-405b-8cc0-be8edfe1bd45 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:02.228633 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2808bb03-04f7-405b-8cc0-be8edfe1bd45 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:02.236147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 631/1260] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:17:02 2022] GET /identity/v3/role_assignments?user.id=fb745b98e3ae4723bad44428646f83e2&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:04.248598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bdb81-5634-4797-a358-5d6cac4d6953 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:04.249159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bdb81-5634-4797-a358-5d6cac4d6953 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:04.249159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6bdb81-5634-4797-a358-5d6cac4d6953 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:04.250525 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 630/1261] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:17:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:17:04.255855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47147c-e059-4e8d-87f8-f2a9c23ded1d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:04.256270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47147c-e059-4e8d-87f8-f2a9c23ded1d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:04.256399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47147c-e059-4e8d-87f8-f2a9c23ded1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:04.277971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0c47147c-e059-4e8d-87f8-f2a9c23ded1d None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:17:04.279221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0c47147c-e059-4e8d-87f8-f2a9c23ded1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:17:04.353541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 632/1262] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:17:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:17:04.650624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bdb81-5634-4797-a358-5d6cac4d6953 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:04.661938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:04.762543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6bdb81-5634-4797-a358-5d6cac4d6953 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:04.801774 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150ccbc-f17d-4e98-bd83-a8fc5fd50204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:04.802335 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150ccbc-f17d-4e98-bd83-a8fc5fd50204 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:04.802609 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150ccbc-f17d-4e98-bd83-a8fc5fd50204 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:04.802872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a150ccbc-f17d-4e98-bd83-a8fc5fd50204 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:04.815441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a150ccbc-f17d-4e98-bd83-a8fc5fd50204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:17:04.820959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-a150ccbc-f17d-4e98-bd83-a8fc5fd50204 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:17:04.820959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:04.820959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:04.820959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:04.820959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:04.820959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:04.820959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:04.820959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:04.820959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:04.820959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:04.820959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:04.820959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:04.820959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:17:04.820959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:17:04.820959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:17:04.820959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:17:04.822066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:17:04.822066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:17:04.822066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:17:04.822066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:17:04.822066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:04.822066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:04.822066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:04.822066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:04.822066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:04.822066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:04.822066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:04.822066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:04.822066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:04.822066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:04.822066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:04.822066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:04.823165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:04.823165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:04.823165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:04.823165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:04.823165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:17:04.823165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:17:04.823165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:17:04.823165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:17:04.823165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:17:04.823165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:17:04.823165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:17:04.823165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:17:04.823165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 631/1263] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:17:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:17:04.830093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47147c-e059-4e8d-87f8-f2a9c23ded1d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:04.851166 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:04.953574 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47147c-e059-4e8d-87f8-f2a9c23ded1d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:04.993029 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290fe5b-64b6-4218-aec7-0a5b1b51229d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:04.993631 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290fe5b-64b6-4218-aec7-0a5b1b51229d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:04.993904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290fe5b-64b6-4218-aec7-0a5b1b51229d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:04.994169 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1290fe5b-64b6-4218-aec7-0a5b1b51229d None admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:04.994857 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290fe5b-64b6-4218-aec7-0a5b1b51229d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:04.995153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290fe5b-64b6-4218-aec7-0a5b1b51229d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:04.996513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1290fe5b-64b6-4218-aec7-0a5b1b51229d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:05.005286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 633/1264] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:17:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:05.011767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150ccbc-f17d-4e98-bd83-a8fc5fd50204 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:05.098204 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:05.134063 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a150ccbc-f17d-4e98-bd83-a8fc5fd50204 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:05.173748 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cc993-c0e8-44a4-8580-88b3e137ceea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:05.174437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cc993-c0e8-44a4-8580-88b3e137ceea None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:05.174713 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cc993-c0e8-44a4-8580-88b3e137ceea None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:05.174979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476cc993-c0e8-44a4-8580-88b3e137ceea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:05.188557 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476cc993-c0e8-44a4-8580-88b3e137ceea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:05.190106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476cc993-c0e8-44a4-8580-88b3e137ceea None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:05.206717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-476cc993-c0e8-44a4-8580-88b3e137ceea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:17:05.206717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:05.206717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:05.206717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:05.206717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:05.206717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:05.206717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:05.206717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:05.206717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:05.206717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:05.206717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:05.206717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:05.206717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:17:05.206717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:17:05.206717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:17:05.206717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:05.207922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:17:05.208832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:17:05.208832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:17:05.208832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:17:05.208832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 632/1265] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:17:05.213887 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290fe5b-64b6-4218-aec7-0a5b1b51229d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:05.297736 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:05.317314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1290fe5b-64b6-4218-aec7-0a5b1b51229d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:05.352189 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a91d3-e8fc-4fe5-b227-bb3d33ba26af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:05.352699 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a91d3-e8fc-4fe5-b227-bb3d33ba26af None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:05.352979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a91d3-e8fc-4fe5-b227-bb3d33ba26af None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:05.353243 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a91d3-e8fc-4fe5-b227-bb3d33ba26af None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:05.353884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a91d3-e8fc-4fe5-b227-bb3d33ba26af None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:05.354179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a91d3-e8fc-4fe5-b227-bb3d33ba26af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:05.355567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a91d3-e8fc-4fe5-b227-bb3d33ba26af None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:05.368188 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 634/1266] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:05.373984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cc993-c0e8-44a4-8580-88b3e137ceea None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:05.471696 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476cc993-c0e8-44a4-8580-88b3e137ceea None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:05.491184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:05.505304 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b18173-9972-4c49-8add-81b4eb7e36e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:05.505956 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b18173-9972-4c49-8add-81b4eb7e36e1 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:05.506243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b18173-9972-4c49-8add-81b4eb7e36e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:05.506522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b18173-9972-4c49-8add-81b4eb7e36e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:05.513974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b18173-9972-4c49-8add-81b4eb7e36e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:05.515410 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b18173-9972-4c49-8add-81b4eb7e36e1 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:05.526321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-d1b18173-9972-4c49-8add-81b4eb7e36e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:17:05.526321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:05.526321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:05.526321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:05.526321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:05.526321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:05.526321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:05.526321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:05.526321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:05.526321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:05.526321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:05.526321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:05.526321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:17:05.526321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:17:05.526321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:17:05.526321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:05.527714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:05.528657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:05.528657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:05.528657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:17:05.528657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:17:05.528657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:17:05.528657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:17:05.528657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:17:05.528657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:17:05.528657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:17:05.528657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:17:05.528657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 633/1267] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:17:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:17:05.533949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a91d3-e8fc-4fe5-b227-bb3d33ba26af None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:05.631992 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a91d3-e8fc-4fe5-b227-bb3d33ba26af None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:05.665706 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425c54c-b596-4e6e-85bb-22e31dbf9b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:05.666323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425c54c-b596-4e6e-85bb-22e31dbf9b57 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:05.667544 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425c54c-b596-4e6e-85bb-22e31dbf9b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:05.667544 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425c54c-b596-4e6e-85bb-22e31dbf9b57 None admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:05.667544 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425c54c-b596-4e6e-85bb-22e31dbf9b57 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:05.667881 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425c54c-b596-4e6e-85bb-22e31dbf9b57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:05.669303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c425c54c-b596-4e6e-85bb-22e31dbf9b57 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:05.678998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:05.687506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 635/1268] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 24 15:17:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:05.693132 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b18173-9972-4c49-8add-81b4eb7e36e1 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:05.791406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b18173-9972-4c49-8add-81b4eb7e36e1 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:05.824656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bed0c6-fc12-45fc-8a21-45d21f5c8357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:05.825269 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bed0c6-fc12-45fc-8a21-45d21f5c8357 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:05.825544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bed0c6-fc12-45fc-8a21-45d21f5c8357 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:05.825810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bed0c6-fc12-45fc-8a21-45d21f5c8357 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:05.836536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bed0c6-fc12-45fc-8a21-45d21f5c8357 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:05.837953 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bed0c6-fc12-45fc-8a21-45d21f5c8357 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:05.852249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-08bed0c6-fc12-45fc-8a21-45d21f5c8357 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:17:05.852249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:05.852249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:05.852249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:05.852249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:05.852249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:05.852249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:05.852249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:05.852249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:05.852249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:05.852249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:05.852249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:05.852249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:17:05.852249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:17:05.852249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:17:05.852249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:05.853570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:17:05.854732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:17:05.854732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:17:05.854732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:17:05.854732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 634/1269] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:17:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:17:05.859222 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425c54c-b596-4e6e-85bb-22e31dbf9b57 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:05.964079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425c54c-b596-4e6e-85bb-22e31dbf9b57 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:05.996354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76f12f-b4b4-44fb-9cf2-c96140cce2ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:05.996953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76f12f-b4b4-44fb-9cf2-c96140cce2ac None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:05.997230 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76f12f-b4b4-44fb-9cf2-c96140cce2ac None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:05.997500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e76f12f-b4b4-44fb-9cf2-c96140cce2ac None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:05.998134 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76f12f-b4b4-44fb-9cf2-c96140cce2ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:05.998459 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76f12f-b4b4-44fb-9cf2-c96140cce2ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:06.000310 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e76f12f-b4b4-44fb-9cf2-c96140cce2ac None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:06.011537 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 636/1270] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:17:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:06.017815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bed0c6-fc12-45fc-8a21-45d21f5c8357 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:06.049088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:06.117206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bed0c6-fc12-45fc-8a21-45d21f5c8357 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:06.150368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac3f8c-5cbb-43a6-ac1a-4d54abdf26eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:06.150988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac3f8c-5cbb-43a6-ac1a-4d54abdf26eb None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:06.151269 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac3f8c-5cbb-43a6-ac1a-4d54abdf26eb None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:06.151539 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac3f8c-5cbb-43a6-ac1a-4d54abdf26eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:06.158845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac3f8c-5cbb-43a6-ac1a-4d54abdf26eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:06.160273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac3f8c-5cbb-43a6-ac1a-4d54abdf26eb None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:06.172554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-47ac3f8c-5cbb-43a6-ac1a-4d54abdf26eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:17:06.172554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:06.172554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:06.172554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:06.172554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:06.172554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:06.172554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:06.172554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:06.172554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:06.172554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:06.172554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:06.172554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:06.172554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:17:06.172554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:17:06.172554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:06.173619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:06.174733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:17:06.174733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:17:06.174733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:17:06.174733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:17:06.174733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:17:06.174733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:17:06.174733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:17:06.174733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:17:06.174733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 635/1271] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:17:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:17:06.179964 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76f12f-b4b4-44fb-9cf2-c96140cce2ac None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:06.238273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:06.282710 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e76f12f-b4b4-44fb-9cf2-c96140cce2ac None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:06.315974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614c990-92c2-4346-ab03-d9d625796d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:06.316614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614c990-92c2-4346-ab03-d9d625796d70 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:06.316833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614c990-92c2-4346-ab03-d9d625796d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:06.317103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614c990-92c2-4346-ab03-d9d625796d70 None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:06.317788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614c990-92c2-4346-ab03-d9d625796d70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:06.318092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614c990-92c2-4346-ab03-d9d625796d70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:06.319547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614c990-92c2-4346-ab03-d9d625796d70 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:06.332839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 637/1272] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:17:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:06.338935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac3f8c-5cbb-43a6-ac1a-4d54abdf26eb None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:06.428451 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:06.437508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac3f8c-5cbb-43a6-ac1a-4d54abdf26eb None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:06.472266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c823d8-8e60-4269-b469-798b55cb5824 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:06.472915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c823d8-8e60-4269-b469-798b55cb5824 None admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:06.473192 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c823d8-8e60-4269-b469-798b55cb5824 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:06.473459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c823d8-8e60-4269-b469-798b55cb5824 None admin] PATH_INFO: `/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/fb745b98e3ae4723bad44428646f83e2/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:06.475472 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c823d8-8e60-4269-b469-798b55cb5824 None admin] RBAC: Authorizing `identity:create_grant(project_id=2c010096b77f4cd3b2f32086b1ba7ea7, user_id=fb745b98e3ae4723bad44428646f83e2, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:06.476902 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c823d8-8e60-4269-b469-798b55cb5824 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:06.489436 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 636/1273] 10.222.0.76 () {58 vars in 1439 bytes} [Wed Aug 24 15:17:06 2022] PUT /identity/v3/projects/2c010096b77f4cd3b2f32086b1ba7ea7/users/fb745b98e3ae4723bad44428646f83e2/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:17:08.470211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2401f0-82a8-4dd0-91cb-dcdc208c1171 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:08.470796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2401f0-82a8-4dd0-91cb-dcdc208c1171 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:08.470796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2401f0-82a8-4dd0-91cb-dcdc208c1171 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:08.472113 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 638/1274] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:17:08.477336 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420672d6-c11d-4d1e-a04a-70fe764077c9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:08.477753 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420672d6-c11d-4d1e-a04a-70fe764077c9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:08.477871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420672d6-c11d-4d1e-a04a-70fe764077c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:08.496248 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-420672d6-c11d-4d1e-a04a-70fe764077c9 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:17:08.497444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-420672d6-c11d-4d1e-a04a-70fe764077c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:17:08.564705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 637/1275] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:17:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:17:08.831026 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2401f0-82a8-4dd0-91cb-dcdc208c1171 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:08.934938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2401f0-82a8-4dd0-91cb-dcdc208c1171 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:08.957692 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:08.976293 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2f6ec-cde8-4317-8c15-4b70b35405c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:08.976919 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2f6ec-cde8-4317-8c15-4b70b35405c7 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:08.977195 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2f6ec-cde8-4317-8c15-4b70b35405c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:08.977451 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2f6ec-cde8-4317-8c15-4b70b35405c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:08.987756 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a2f6ec-cde8-4317-8c15-4b70b35405c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 15:17:08.992389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-a7a2f6ec-cde8-4317-8c15-4b70b35405c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:17:08.992389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:08.992389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:08.992389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:08.992389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:08.992389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:08.992389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:08.992389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:08.992389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:08.992389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:08.992389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:08.992389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:08.992389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 15:17:08.992389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 15:17:08.992389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 15:17:08.992389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 15:17:08.993379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 15:17:08.993379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 15:17:08.993379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 15:17:08.993379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 15:17:08.993379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:08.993379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:08.993379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:08.993379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:08.993379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:08.993379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:08.993379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:08.993379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:08.993379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:08.993379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:08.993379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:08.993379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:08.994396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:08.994396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:08.994396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:08.994396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:08.994396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 15:17:08.994396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 15:17:08.994396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 15:17:08.994396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 15:17:08.994396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 15:17:08.994396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 15:17:08.994396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 15:17:08.994396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:17:08.994396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 639/1276] 10.222.0.76 () {56 vars in 1096 bytes} [Wed Aug 24 15:17:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:17:08.999612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420672d6-c11d-4d1e-a04a-70fe764077c9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:09.079564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:09.099507 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420672d6-c11d-4d1e-a04a-70fe764077c9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:09.134266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8554246-d211-477f-8bfd-79902946222f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:09.134874 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8554246-d211-477f-8bfd-79902946222f None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:09.135155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8554246-d211-477f-8bfd-79902946222f None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:09.135426 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8554246-d211-477f-8bfd-79902946222f None admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:09.136088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8554246-d211-477f-8bfd-79902946222f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:09.136387 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8554246-d211-477f-8bfd-79902946222f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:09.137744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8554246-d211-477f-8bfd-79902946222f None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:09.145876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 638/1277] 10.222.0.76 () {56 vars in 1099 bytes} [Wed Aug 24 15:17:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:09.152344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2f6ec-cde8-4317-8c15-4b70b35405c7 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:09.251968 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a2f6ec-cde8-4317-8c15-4b70b35405c7 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:09.285303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38737b-96d4-41ce-8ff6-8ff183362bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:09.285917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38737b-96d4-41ce-8ff6-8ff183362bfb None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:09.286193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38737b-96d4-41ce-8ff6-8ff183362bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:09.286495 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38737b-96d4-41ce-8ff6-8ff183362bfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:09.296125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38737b-96d4-41ce-8ff6-8ff183362bfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:09.298162 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38737b-96d4-41ce-8ff6-8ff183362bfb None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:09.312902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-5f38737b-96d4-41ce-8ff6-8ff183362bfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:17:09.312902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:09.312902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:09.312902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:09.312902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:09.312902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:09.312902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:09.312902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:09.312902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:09.312902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:09.312902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:09.312902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:09.312902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:17:09.312902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:17:09.312902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:17:09.312902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:09.314008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:17:09.314997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:17:09.314997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:17:09.314997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:17:09.314997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 640/1278] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:17:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:17:09.320184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8554246-d211-477f-8bfd-79902946222f None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:09.419424 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8554246-d211-477f-8bfd-79902946222f None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:09.452084 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fb112-495b-40b6-bb1f-058bfa1caf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:09.452686 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fb112-495b-40b6-bb1f-058bfa1caf1b None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:09.453566 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fb112-495b-40b6-bb1f-058bfa1caf1b None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:09.453838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fb112-495b-40b6-bb1f-058bfa1caf1b None admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:09.454514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fb112-495b-40b6-bb1f-058bfa1caf1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:09.454817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fb112-495b-40b6-bb1f-058bfa1caf1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:09.456172 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fb112-495b-40b6-bb1f-058bfa1caf1b None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:09.467728 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 639/1279] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:17:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:09.473986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38737b-96d4-41ce-8ff6-8ff183362bfb None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:09.505889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:09.575925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38737b-96d4-41ce-8ff6-8ff183362bfb None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:09.610815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bee75d-31bd-4a72-ac3c-22cdcaf1b99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:09.611452 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bee75d-31bd-4a72-ac3c-22cdcaf1b99d None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:09.611731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bee75d-31bd-4a72-ac3c-22cdcaf1b99d None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:09.612003 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bee75d-31bd-4a72-ac3c-22cdcaf1b99d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:09.620117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bee75d-31bd-4a72-ac3c-22cdcaf1b99d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:09.621594 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bee75d-31bd-4a72-ac3c-22cdcaf1b99d None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:09.633249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-c6bee75d-31bd-4a72-ac3c-22cdcaf1b99d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:17:09.633249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:09.633249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:09.633249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:09.633249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:09.633249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:09.633249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:09.633249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:09.633249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:09.633249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:09.633249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:09.633249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:09.633249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 15:17:09.633249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 15:17:09.633249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 15:17:09.633249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:09.634449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:09.635721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:09.635721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:09.635721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 15:17:09.635721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 15:17:09.635721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 15:17:09.635721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 15:17:09.635721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 15:17:09.635721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 15:17:09.635721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 15:17:09.635721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:17:09.635721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 641/1280] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:17:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:17:09.640967 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fb112-495b-40b6-bb1f-058bfa1caf1b None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:09.646103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:09.746831 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fb112-495b-40b6-bb1f-058bfa1caf1b None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:09.781505 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a629e0-fe70-4301-b922-9c52df5efa3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:09.782100 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a629e0-fe70-4301-b922-9c52df5efa3c None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:09.782413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a629e0-fe70-4301-b922-9c52df5efa3c None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:09.782684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a629e0-fe70-4301-b922-9c52df5efa3c None admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:09.783415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a629e0-fe70-4301-b922-9c52df5efa3c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:09.783720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a629e0-fe70-4301-b922-9c52df5efa3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:09.785137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a629e0-fe70-4301-b922-9c52df5efa3c None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:09.803476 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 640/1281] 10.222.0.76 () {56 vars in 1143 bytes} [Wed Aug 24 15:17:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:09.809188 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bee75d-31bd-4a72-ac3c-22cdcaf1b99d None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:09.902975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:09.913240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bee75d-31bd-4a72-ac3c-22cdcaf1b99d None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:09.946599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d516cc-8f52-4438-82c3-feb7777b1df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:09.947189 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d516cc-8f52-4438-82c3-feb7777b1df3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:09.947463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d516cc-8f52-4438-82c3-feb7777b1df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:09.947728 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d516cc-8f52-4438-82c3-feb7777b1df3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:10.017457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d516cc-8f52-4438-82c3-feb7777b1df3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:10.018841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d516cc-8f52-4438-82c3-feb7777b1df3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:10.103424 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:10.114498 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 642/1282] 10.222.0.76 () {58 vars in 1282 bytes} [Wed Aug 24 15:17:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 15:17:10.120092 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a629e0-fe70-4301-b922-9c52df5efa3c None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:10.179298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:10.217888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a629e0-fe70-4301-b922-9c52df5efa3c None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:10.251414 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ac3d4-d72c-4215-843d-9d8a56a48883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:10.252033 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ac3d4-d72c-4215-843d-9d8a56a48883 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:10.252305 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ac3d4-d72c-4215-843d-9d8a56a48883 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:10.252570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0ac3d4-d72c-4215-843d-9d8a56a48883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:10.262162 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ac3d4-d72c-4215-843d-9d8a56a48883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:10.263600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0ac3d4-d72c-4215-843d-9d8a56a48883 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:10.277945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-0c0ac3d4-d72c-4215-843d-9d8a56a48883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:17:10.277945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:10.277945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:10.277945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:10.277945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:10.277945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:10.277945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:10.277945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:10.277945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:10.277945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:10.277945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:10.277945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:10.277945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 15:17:10.277945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 15:17:10.277945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 15:17:10.277945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:10.279024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 15:17:10.280238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 15:17:10.280238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 15:17:10.280238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:17:10.280238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 641/1283] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:17:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:17:10.284821 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d516cc-8f52-4438-82c3-feb7777b1df3 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:10.380947 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d516cc-8f52-4438-82c3-feb7777b1df3 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:10.413420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd81e7-016b-498e-9f04-80028dfdab01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:10.414009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd81e7-016b-498e-9f04-80028dfdab01 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:10.414315 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd81e7-016b-498e-9f04-80028dfdab01 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:10.414585 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd81e7-016b-498e-9f04-80028dfdab01 None admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:10.415209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd81e7-016b-498e-9f04-80028dfdab01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:10.415502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd81e7-016b-498e-9f04-80028dfdab01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:10.416853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd81e7-016b-498e-9f04-80028dfdab01 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:10.428246 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 643/1284] 10.222.0.76 () {56 vars in 1109 bytes} [Wed Aug 24 15:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:10.433914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ac3d4-d72c-4215-843d-9d8a56a48883 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:10.533026 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0ac3d4-d72c-4215-843d-9d8a56a48883 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:10.546377 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:10.566650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e9c75-b8ea-4769-8771-1d861b370055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:10.567272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e9c75-b8ea-4769-8771-1d861b370055 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:10.567591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e9c75-b8ea-4769-8771-1d861b370055 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:10.567896 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448e9c75-b8ea-4769-8771-1d861b370055 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:10.574359 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e9c75-b8ea-4769-8771-1d861b370055 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:10.575786 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448e9c75-b8ea-4769-8771-1d861b370055 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:10.586625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application [None req-448e9c75-b8ea-4769-8771-1d861b370055 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:17:10.586625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:10.586625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:10.586625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:10.586625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:10.586625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:10.586625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:10.586625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:10.586625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:10.586625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:10.586625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:10.586625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:10.586625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 15:17:10.586625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 15:17:10.586625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:10.587689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:10.588801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 15:17:10.588801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 15:17:10.588801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 15:17:10.588801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 15:17:10.588801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 15:17:10.588801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 15:17:10.588801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 15:17:10.588801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: ERROR keystone.server.flask.application Aug 24 15:17:10.588801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 642/1285] 10.222.0.76 () {56 vars in 1111 bytes} [Wed Aug 24 15:17:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:17:10.594011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd81e7-016b-498e-9f04-80028dfdab01 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:10.641700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:10.690881 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd81e7-016b-498e-9f04-80028dfdab01 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:10.722803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66f7f8-6652-432a-a9c1-e076f1a4ffee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:10.723391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66f7f8-6652-432a-a9c1-e076f1a4ffee None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:10.723662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66f7f8-6652-432a-a9c1-e076f1a4ffee None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:10.724078 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66f7f8-6652-432a-a9c1-e076f1a4ffee None admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:10.724596 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66f7f8-6652-432a-a9c1-e076f1a4ffee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:10.724895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66f7f8-6652-432a-a9c1-e076f1a4ffee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:10.726332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66f7f8-6652-432a-a9c1-e076f1a4ffee None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:10.738357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 644/1286] 10.222.0.76 () {56 vars in 1148 bytes} [Wed Aug 24 15:17:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:10.744362 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e9c75-b8ea-4769-8771-1d861b370055 None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:10.841063 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448e9c75-b8ea-4769-8771-1d861b370055 None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:10.873384 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9910b228-7695-4f7c-82f1-1ed6d5dd9cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:10.873972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910b228-7695-4f7c-82f1-1ed6d5dd9cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:10.874272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910b228-7695-4f7c-82f1-1ed6d5dd9cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:10.874546 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9910b228-7695-4f7c-82f1-1ed6d5dd9cc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:10.875283 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910b228-7695-4f7c-82f1-1ed6d5dd9cc6 None admin] RBAC: Adding query filter params (role.id=b00780d2bbfb4c12b9bcab9312340187, scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7, user.id=fb745b98e3ae4723bad44428646f83e2) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:10.875587 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910b228-7695-4f7c-82f1-1ed6d5dd9cc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:10.877000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9910b228-7695-4f7c-82f1-1ed6d5dd9cc6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:10.883143 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 643/1287] 10.222.0.76 () {56 vars in 1374 bytes} [Wed Aug 24 15:17:10 2022] GET /identity/v3/role_assignments?user.id=fb745b98e3ae4723bad44428646f83e2&scope.project.id=2c010096b77f4cd3b2f32086b1ba7ea7&role.id=b00780d2bbfb4c12b9bcab9312340187 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:12.785261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f67e7-f968-4b2b-ac87-71b52f737f62 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:12.785802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f67e7-f968-4b2b-ac87-71b52f737f62 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:12.785802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392f67e7-f968-4b2b-ac87-71b52f737f62 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:12.787154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 645/1288] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:17:12.792413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68486b91-eadc-44b3-98bc-2c85263be36d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:12.792789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68486b91-eadc-44b3-98bc-2c85263be36d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:12.792957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68486b91-eadc-44b3-98bc-2c85263be36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:12.811289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-68486b91-eadc-44b3-98bc-2c85263be36d None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:17:12.812496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-68486b91-eadc-44b3-98bc-2c85263be36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:17:12.840793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:12.877664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 644/1289] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:17:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:17:13.142062 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f67e7-f968-4b2b-ac87-71b52f737f62 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:13.248885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392f67e7-f968-4b2b-ac87-71b52f737f62 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:13.259788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:13.283563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a01c2c-ec9d-4385-acca-786e5130d084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:13.284216 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a01c2c-ec9d-4385-acca-786e5130d084 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:13.284492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a01c2c-ec9d-4385-acca-786e5130d084 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:13.284760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a01c2c-ec9d-4385-acca-786e5130d084 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:13.285332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a01c2c-ec9d-4385-acca-786e5130d084 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:13.286756 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a01c2c-ec9d-4385-acca-786e5130d084 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-57a01c2c-ec9d-4385-acca-786e5130d084 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:13.295274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:13.296933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:13.296933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:13.296933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:13.296933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:13.296933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:13.296933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:13.296933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:17:13.296933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:17:13.296933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:17:13.296933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:17:13.296933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:13.296933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:13.296933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:13.296933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:13.296933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:13.296933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:13.298152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:13.298152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:13.298152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:13.298152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:13.298152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:13.298152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:13.298152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:13.298152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:13.298152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:13.298152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:13.298152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:17:13.298152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:17:13.298152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:17:13.298152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:17:13.298152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 646/1290] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 15:17:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:17:13.302899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68486b91-eadc-44b3-98bc-2c85263be36d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:13.399622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68486b91-eadc-44b3-98bc-2c85263be36d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:13.433383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169bd38e-b8ff-4bb2-a6ef-8b8510988f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:13.433979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bd38e-b8ff-4bb2-a6ef-8b8510988f07 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:13.434278 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bd38e-b8ff-4bb2-a6ef-8b8510988f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:13.434551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bd38e-b8ff-4bb2-a6ef-8b8510988f07 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:13.435172 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169bd38e-b8ff-4bb2-a6ef-8b8510988f07 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:13.435468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169bd38e-b8ff-4bb2-a6ef-8b8510988f07 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:13.436814 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169bd38e-b8ff-4bb2-a6ef-8b8510988f07 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:13.442028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 645/1291] 10.222.0.76 () {56 vars in 1116 bytes} [Wed Aug 24 15:17:13 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:13.448005 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a01c2c-ec9d-4385-acca-786e5130d084 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:13.550252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a01c2c-ec9d-4385-acca-786e5130d084 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:13.583826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3884b77f-6265-4124-ad9f-03f7d753301c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:13.584424 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884b77f-6265-4124-ad9f-03f7d753301c None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:13.584704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884b77f-6265-4124-ad9f-03f7d753301c None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:13.584977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3884b77f-6265-4124-ad9f-03f7d753301c None admin] PATH_INFO: `/v3/services` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:13.585599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884b77f-6265-4124-ad9f-03f7d753301c None admin] RBAC: Adding query filter params (type=placement) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:13.585900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884b77f-6265-4124-ad9f-03f7d753301c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:13.587295 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3884b77f-6265-4124-ad9f-03f7d753301c None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:13.591678 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 647/1292] 10.222.0.76 () {56 vars in 1116 bytes} [Wed Aug 24 15:17:13 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:15.473475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c086a-7424-41ca-af86-90909dc24ed2 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:15.474012 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c086a-7424-41ca-af86-90909dc24ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:15.474012 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9c086a-7424-41ca-af86-90909dc24ed2 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:15.475473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 646/1293] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:17:15.480656 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88215d-ec06-4bfd-b517-3d7d54193ec6 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:15.481054 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88215d-ec06-4bfd-b517-3d7d54193ec6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:15.481211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d88215d-ec06-4bfd-b517-3d7d54193ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:15.504025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6d88215d-ec06-4bfd-b517-3d7d54193ec6 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:17:15.505229 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6d88215d-ec06-4bfd-b517-3d7d54193ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:17:15.534515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:15.572581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 648/1294] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:17:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:17:15.848596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c086a-7424-41ca-af86-90909dc24ed2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:15.879569 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:15.962414 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9c086a-7424-41ca-af86-90909dc24ed2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:15.996638 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b07480a-4096-4ac7-8a3f-51bc4f6eb8e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:15.997228 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07480a-4096-4ac7-8a3f-51bc4f6eb8e6 None admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:15.997503 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07480a-4096-4ac7-8a3f-51bc4f6eb8e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:15.997771 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b07480a-4096-4ac7-8a3f-51bc4f6eb8e6 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:15.998437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07480a-4096-4ac7-8a3f-51bc4f6eb8e6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:15.999822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b07480a-4096-4ac7-8a3f-51bc4f6eb8e6 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:16.013818 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 647/1295] 10.222.0.76 () {60 vars in 1141 bytes} [Wed Aug 24 15:17:15 2022] POST /identity/v3/services => generated 246 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:17:17.889885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61a2c0-5570-4a28-9d36-2f84fc2e5971 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:17.890939 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61a2c0-5570-4a28-9d36-2f84fc2e5971 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:17.890939 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61a2c0-5570-4a28-9d36-2f84fc2e5971 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:17.892221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 649/1296] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:17:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:17:17.897373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e631a-28f4-4180-9fd0-fb93f01c0603 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:17.897767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e631a-28f4-4180-9fd0-fb93f01c0603 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:17.897911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e631a-28f4-4180-9fd0-fb93f01c0603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:17.916478 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-500e631a-28f4-4180-9fd0-fb93f01c0603 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:17:17.917650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-500e631a-28f4-4180-9fd0-fb93f01c0603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:17:17.988169 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 648/1297] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:17:17 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 15:17:18.252708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61a2c0-5570-4a28-9d36-2f84fc2e5971 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:18.351081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61a2c0-5570-4a28-9d36-2f84fc2e5971 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:18.384195 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca517e66-a2f1-4e59-b0f8-e5b70aafa85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:18.384806 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca517e66-a2f1-4e59-b0f8-e5b70aafa85a None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:18.385116 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca517e66-a2f1-4e59-b0f8-e5b70aafa85a None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:18.385383 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca517e66-a2f1-4e59-b0f8-e5b70aafa85a None admin] PATH_INFO: `/v3/services/placement` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:18.385954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca517e66-a2f1-4e59-b0f8-e5b70aafa85a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:18.387340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca517e66-a2f1-4e59-b0f8-e5b70aafa85a None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-ca517e66-a2f1-4e59-b0f8-e5b70aafa85a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:18.395807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:18.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:18.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:18.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:18.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:18.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:18.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:18.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:17:18.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:17:18.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:17:18.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:17:18.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:18.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:18.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:18.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:18.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:18.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:18.398038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:18.398038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:18.398038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:18.398038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:18.398038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:18.398038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:18.398038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:18.398038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:18.398038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:18.398038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:18.398038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:17:18.398038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:17:18.398038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:17:18.398038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:17:18.398038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 650/1298] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 15:17:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:17:18.403082 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e631a-28f4-4180-9fd0-fb93f01c0603 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:18.420868 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:18.503093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e631a-28f4-4180-9fd0-fb93f01c0603 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:18.536942 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914ed48-974f-44d6-a581-88e07afc116d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:18.537546 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914ed48-974f-44d6-a581-88e07afc116d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:18.537822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914ed48-974f-44d6-a581-88e07afc116d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:18.538089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9914ed48-974f-44d6-a581-88e07afc116d None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:18.538746 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914ed48-974f-44d6-a581-88e07afc116d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:18.539045 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914ed48-974f-44d6-a581-88e07afc116d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:18.540400 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9914ed48-974f-44d6-a581-88e07afc116d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:18.545415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 649/1299] 10.222.0.76 () {56 vars in 1116 bytes} [Wed Aug 24 15:17:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:18.551917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca517e66-a2f1-4e59-b0f8-e5b70aafa85a None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:18.582818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:18.650500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca517e66-a2f1-4e59-b0f8-e5b70aafa85a None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:18.683299 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c3c137-c2dc-4588-b86b-d8b346b088b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:18.683887 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3c137-c2dc-4588-b86b-d8b346b088b3 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:18.684164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3c137-c2dc-4588-b86b-d8b346b088b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:18.684435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c3c137-c2dc-4588-b86b-d8b346b088b3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:18.685125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3c137-c2dc-4588-b86b-d8b346b088b3 None admin] RBAC: Adding query filter params (interface=public, service_id=da44a8095e8444c186ed8edc65d8d643, region_id=RegionOne) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:18.685428 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3c137-c2dc-4588-b86b-d8b346b088b3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:18.686828 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c3c137-c2dc-4588-b86b-d8b346b088b3 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:18.692306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 651/1300] 10.222.0.76 () {56 vars in 1251 bytes} [Wed Aug 24 15:17:18 2022] GET /identity/v3/endpoints?service_id=da44a8095e8444c186ed8edc65d8d643&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:18.698133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914ed48-974f-44d6-a581-88e07afc116d None admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:18.776521 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:18.794726 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9914ed48-974f-44d6-a581-88e07afc116d None admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:18.828393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7349b8f0-f8b4-468d-89af-dbf7e4d25aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:18.828985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349b8f0-f8b4-468d-89af-dbf7e4d25aa0 None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:18.829261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349b8f0-f8b4-468d-89af-dbf7e4d25aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:18.829532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7349b8f0-f8b4-468d-89af-dbf7e4d25aa0 None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:18.830122 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349b8f0-f8b4-468d-89af-dbf7e4d25aa0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:18.831508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7349b8f0-f8b4-468d-89af-dbf7e4d25aa0 None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:18.838235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 650/1301] 10.222.0.76 () {56 vars in 1087 bytes} [Wed Aug 24 15:17:18 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:17:20.708156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070801b5-16f8-446e-ae3a-e0696be434d3 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:20.708714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070801b5-16f8-446e-ae3a-e0696be434d3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:20.708714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070801b5-16f8-446e-ae3a-e0696be434d3 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:20.710022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 652/1302] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:17:20.715493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873efa7f-0fe8-4198-b43f-86c3dc941b73 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:20.715890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873efa7f-0fe8-4198-b43f-86c3dc941b73 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:20.716028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873efa7f-0fe8-4198-b43f-86c3dc941b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:20.734133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-873efa7f-0fe8-4198-b43f-86c3dc941b73 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:17:20.735328 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-873efa7f-0fe8-4198-b43f-86c3dc941b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:17:20.800347 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 651/1303] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:17:20 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 15:17:21.081052 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070801b5-16f8-446e-ae3a-e0696be434d3 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:21.174452 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:21.181479 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070801b5-16f8-446e-ae3a-e0696be434d3 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:21.214667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256682c6-7fe9-4b53-bade-da4911802f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:21.215282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256682c6-7fe9-4b53-bade-da4911802f53 None admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:21.215559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256682c6-7fe9-4b53-bade-da4911802f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:21.215825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256682c6-7fe9-4b53-bade-da4911802f53 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:21.216391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256682c6-7fe9-4b53-bade-da4911802f53 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:21.217756 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256682c6-7fe9-4b53-bade-da4911802f53 None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application [None req-256682c6-7fe9-4b53-bade-da4911802f53 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 15:17:21.226235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 15:17:21.227395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 15:17:21.227395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 15:17:21.227395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 15:17:21.227395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 15:17:21.227395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 15:17:21.227395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 15:17:21.227395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 15:17:21.227395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 15:17:21.227395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 15:17:21.227395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 15:17:21.227395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 15:17:21.227395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 15:17:21.227395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 15:17:21.227395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 15:17:21.227395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 15:17:21.227395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 15:17:21.228539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 15:17:21.228539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application with Lock( Aug 24 15:17:21.228539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 15:17:21.228539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self._enter() Aug 24 15:17:21.228539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 15:17:21.228539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 15:17:21.228539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 15:17:21.228539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application return self.creator() Aug 24 15:17:21.228539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 15:17:21.228539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application created_value = creator( Aug 24 15:17:21.228539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 15:17:21.228539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 15:17:21.228539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 15:17:21.228539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: ERROR keystone.server.flask.application Aug 24 15:17:21.228539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 653/1304] 10.222.0.76 () {56 vars in 1117 bytes} [Wed Aug 24 15:17:21 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 15:17:21.235832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873efa7f-0fe8-4198-b43f-86c3dc941b73 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:21.308442 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:21.331648 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873efa7f-0fe8-4198-b43f-86c3dc941b73 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:21.365469 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f7bade-5fba-43a5-8c4c-cf90ba3eb04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:21.366068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7bade-5fba-43a5-8c4c-cf90ba3eb04d None admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:21.366383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7bade-5fba-43a5-8c4c-cf90ba3eb04d None admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:21.366654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f7bade-5fba-43a5-8c4c-cf90ba3eb04d None admin] PATH_INFO: `/v3/services` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:21.367278 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7bade-5fba-43a5-8c4c-cf90ba3eb04d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:17:21.367577 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7bade-5fba-43a5-8c4c-cf90ba3eb04d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:21.368933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f7bade-5fba-43a5-8c4c-cf90ba3eb04d None admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:21.374094 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 652/1305] 10.222.0.76 () {56 vars in 1116 bytes} [Wed Aug 24 15:17:21 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:17:21.380469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256682c6-7fe9-4b53-bade-da4911802f53 None admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:17:21.477726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256682c6-7fe9-4b53-bade-da4911802f53 None admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:17:21.512557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c6c062-5552-4a8f-84eb-12c15ede21cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:17:21.513155 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6c062-5552-4a8f-84eb-12c15ede21cf None admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:17:21.513434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6c062-5552-4a8f-84eb-12c15ede21cf None admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:17:21.513702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c6c062-5552-4a8f-84eb-12c15ede21cf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:17:21.514382 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6c062-5552-4a8f-84eb-12c15ede21cf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:17:21.515865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c6c062-5552-4a8f-84eb-12c15ede21cf None admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:17:21.537048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 654/1306] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:17:21 2022] POST /identity/v3/endpoints => generated 331 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:17:31.372340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=30 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:17:31.533467 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:27.671542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a18cd-d3cf-446b-a403-bc99f9f184c9 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:27.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a18cd-d3cf-446b-a403-bc99f9f184c9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:27.672106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9a18cd-d3cf-446b-a403-bc99f9f184c9 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:27.677930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 653/1307] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:18:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:18:27.679236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080746a4-c8f6-4456-97a9-66d54b2096ba None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:27.680068 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080746a4-c8f6-4456-97a9-66d54b2096ba None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:27.680552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080746a4-c8f6-4456-97a9-66d54b2096ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:27.769280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-080746a4-c8f6-4456-97a9-66d54b2096ba None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:18:27.770473 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-080746a4-c8f6-4456-97a9-66d54b2096ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:27.890603 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 655/1308] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:18:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:18:28.177791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d02ed-2d54-4466-9e1c-997e9cebb8a7 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:28.178243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d02ed-2d54-4466-9e1c-997e9cebb8a7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:28.178598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347d02ed-2d54-4466-9e1c-997e9cebb8a7 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:28.180031 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 654/1309] 10.222.0.76 () {54 vars in 965 bytes} [Wed Aug 24 15:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:18:28.186455 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd7fbe-a898-47d8-98ee-ccc86d91fd8f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:28.186747 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd7fbe-a898-47d8-98ee-ccc86d91fd8f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:28.187022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd7fbe-a898-47d8-98ee-ccc86d91fd8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:28.282139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-25cd7fbe-a898-47d8-98ee-ccc86d91fd8f None None] MFA Rules not processed for user `5427bd0ac88042338b956c10cfd07baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:18:28.284248 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-25cd7fbe-a898-47d8-98ee-ccc86d91fd8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:28.309502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:28.373464 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 656/1310] 10.222.0.76 () {58 vars in 1062 bytes} [Wed Aug 24 15:18:28 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:18:28.382595 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d02ed-2d54-4466-9e1c-997e9cebb8a7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:18:28.384468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-347d02ed-2d54-4466-9e1c-997e9cebb8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:28.511778 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347d02ed-2d54-4466-9e1c-997e9cebb8a7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:18:28.595117 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918adaa-1ea8-4103-b9d7-8b7fa8e2d8ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:18:28.595632 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918adaa-1ea8-4103-b9d7-8b7fa8e2d8ca service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:28.595910 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918adaa-1ea8-4103-b9d7-8b7fa8e2d8ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:28.596535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918adaa-1ea8-4103-b9d7-8b7fa8e2d8ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:28.597914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6918adaa-1ea8-4103-b9d7-8b7fa8e2d8ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:28.667334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:28.730080 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918adaa-1ea8-4103-b9d7-8b7fa8e2d8ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:18:28.731965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918adaa-1ea8-4103-b9d7-8b7fa8e2d8ca service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:18:28.755268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 655/1311] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:18:31.023305 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633af17-0e2a-43c0-8e69-ab9d04b06d5a None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:31.023863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633af17-0e2a-43c0-8e69-ab9d04b06d5a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:31.023863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7633af17-0e2a-43c0-8e69-ab9d04b06d5a None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:31.026171 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 657/1312] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:18:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:18:31.032692 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c3953-e5f9-4be5-af3a-6fb8301f44f8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:31.033296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c3953-e5f9-4be5-af3a-6fb8301f44f8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:31.033296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c3953-e5f9-4be5-af3a-6fb8301f44f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:31.121468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8c6c3953-e5f9-4be5-af3a-6fb8301f44f8 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:18:31.123207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8c6c3953-e5f9-4be5-af3a-6fb8301f44f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:31.234657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 656/1313] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:18:31.538284 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e7e7d-c4c0-427f-b7df-ccdf4dd7e57e None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:31.538573 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e7e7d-c4c0-427f-b7df-ccdf4dd7e57e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:31.538872 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315e7e7d-c4c0-427f-b7df-ccdf4dd7e57e None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:31.540240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 658/1314] 10.222.0.76 () {54 vars in 965 bytes} [Wed Aug 24 15:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:18:31.546938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662032b2-67da-4137-8104-0bbdf3b3e530 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:31.547299 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662032b2-67da-4137-8104-0bbdf3b3e530 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:31.547578 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662032b2-67da-4137-8104-0bbdf3b3e530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:31.559443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:31.624559 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-662032b2-67da-4137-8104-0bbdf3b3e530 None None] MFA Rules not processed for user `5427bd0ac88042338b956c10cfd07baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:18:31.625685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-662032b2-67da-4137-8104-0bbdf3b3e530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:31.708190 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 657/1315] 10.222.0.76 () {58 vars in 1062 bytes} [Wed Aug 24 15:18:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:18:31.717636 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e7e7d-c4c0-427f-b7df-ccdf4dd7e57e None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:18:31.719214 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-315e7e7d-c4c0-427f-b7df-ccdf4dd7e57e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:31.828543 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315e7e7d-c4c0-427f-b7df-ccdf4dd7e57e None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:18:31.904582 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:31.907617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc87ffe-690e-4e0b-8cdc-f8dcbad346a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:18:31.908415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc87ffe-690e-4e0b-8cdc-f8dcbad346a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:31.908900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc87ffe-690e-4e0b-8cdc-f8dcbad346a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:31.909180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc87ffe-690e-4e0b-8cdc-f8dcbad346a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:31.910856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ddc87ffe-690e-4e0b-8cdc-f8dcbad346a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:32.024398 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc87ffe-690e-4e0b-8cdc-f8dcbad346a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:18:32.025852 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc87ffe-690e-4e0b-8cdc-f8dcbad346a5 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:18:32.045639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 659/1316] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:18:34.499542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c226c-d095-4d63-b695-56c05a671251 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:34.500110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c226c-d095-4d63-b695-56c05a671251 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:34.500110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c226c-d095-4d63-b695-56c05a671251 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:34.501440 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 658/1317] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:18:34.506969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bac4ce-91f6-4abf-afd8-80f5488baf81 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:34.507373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bac4ce-91f6-4abf-afd8-80f5488baf81 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:34.507527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bac4ce-91f6-4abf-afd8-80f5488baf81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:34.572684 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a2bac4ce-91f6-4abf-afd8-80f5488baf81 None None] MFA Rules not processed for user `179374916edf4dd480273f6c8c748ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:18:34.573889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a2bac4ce-91f6-4abf-afd8-80f5488baf81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:34.625998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:34.673099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 660/1318] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:18:34 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:18:34.959596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c226c-d095-4d63-b695-56c05a671251 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:18:34.961146 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a64c226c-d095-4d63-b695-56c05a671251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:35.065451 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c226c-d095-4d63-b695-56c05a671251 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:18:35.068355 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:35.140613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a706ae94-8634-4692-a90a-fda749e85cd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:18:35.141216 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706ae94-8634-4692-a90a-fda749e85cd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:35.141493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706ae94-8634-4692-a90a-fda749e85cd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:35.141762 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a706ae94-8634-4692-a90a-fda749e85cd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:35.143397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a706ae94-8634-4692-a90a-fda749e85cd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:35.252838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706ae94-8634-4692-a90a-fda749e85cd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:18:35.254186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a706ae94-8634-4692-a90a-fda749e85cd6 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:18:35.274101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 659/1319] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:18:34 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:18:37.732742 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee50a9-24df-469e-b096-32c635c66a47 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:37.733346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee50a9-24df-469e-b096-32c635c66a47 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:37.733346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee50a9-24df-469e-b096-32c635c66a47 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:37.734659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 661/1320] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:18:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:18:37.740019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb6972-7b4b-488b-b4af-1e6d0e8a657b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:37.740418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb6972-7b4b-488b-b4af-1e6d0e8a657b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:37.740564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8eb6972-7b4b-488b-b4af-1e6d0e8a657b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:37.781419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:37.807114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c8eb6972-7b4b-488b-b4af-1e6d0e8a657b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:18:37.808346 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c8eb6972-7b4b-488b-b4af-1e6d0e8a657b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:37.906669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 660/1321] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:18:37 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:18:38.191907 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee50a9-24df-469e-b096-32c635c66a47 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:18:38.193449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-efee50a9-24df-469e-b096-32c635c66a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:38.297288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee50a9-24df-469e-b096-32c635c66a47 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:18:38.370149 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2ede7f-0171-4c48-9666-e99be426442c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:18:38.370782 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ede7f-0171-4c48-9666-e99be426442c service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:38.371058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ede7f-0171-4c48-9666-e99be426442c service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:38.371327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2ede7f-0171-4c48-9666-e99be426442c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:38.373453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9c2ede7f-0171-4c48-9666-e99be426442c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:38.380842 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:38.483168 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ede7f-0171-4c48-9666-e99be426442c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:18:38.484511 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2ede7f-0171-4c48-9666-e99be426442c service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:18:38.503061 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 662/1322] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:18:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:18:41.389358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1ab53-e003-4d85-8be7-8fdbb19b37cf None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:41.389939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1ab53-e003-4d85-8be7-8fdbb19b37cf None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:41.390033 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1ab53-e003-4d85-8be7-8fdbb19b37cf None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:41.391956 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 661/1323] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:18:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:18:41.398309 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac0e25-4c0a-4234-9962-20c29b28b989 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:41.398883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac0e25-4c0a-4234-9962-20c29b28b989 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:41.398883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac0e25-4c0a-4234-9962-20c29b28b989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:41.472956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cdac0e25-4c0a-4234-9962-20c29b28b989 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:18:41.474235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cdac0e25-4c0a-4234-9962-20c29b28b989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:41.538191 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:41.581374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 663/1324] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:18:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:18:41.862273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1ab53-e003-4d85-8be7-8fdbb19b37cf None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:18:41.863869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fdd1ab53-e003-4d85-8be7-8fdbb19b37cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:41.976284 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1ab53-e003-4d85-8be7-8fdbb19b37cf None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:18:41.989410 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:42.055843 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e3908-7dad-4803-9b50-907e73713d9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:18:42.056458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e3908-7dad-4803-9b50-907e73713d9e service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:42.056742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e3908-7dad-4803-9b50-907e73713d9e service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:42.057063 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e3908-7dad-4803-9b50-907e73713d9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:42.058681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-507e3908-7dad-4803-9b50-907e73713d9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:42.169552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e3908-7dad-4803-9b50-907e73713d9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:18:42.171005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e3908-7dad-4803-9b50-907e73713d9e service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:18:42.189343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 662/1325] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:18:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:18:44.161052 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c69148-3350-4a54-9621-a1996fadd8d3 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:44.161599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c69148-3350-4a54-9621-a1996fadd8d3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:44.161882 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c69148-3350-4a54-9621-a1996fadd8d3 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:44.163264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 664/1326] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:18:44.169024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbcaa5-8bf3-4cd3-8291-3e23396b7c8b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:44.169453 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbcaa5-8bf3-4cd3-8291-3e23396b7c8b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:44.169546 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75cbcaa5-8bf3-4cd3-8291-3e23396b7c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:44.219106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:44.239878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-75cbcaa5-8bf3-4cd3-8291-3e23396b7c8b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:18:44.241123 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-75cbcaa5-8bf3-4cd3-8291-3e23396b7c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:44.355236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 663/1327] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:18:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:18:44.639410 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c69148-3350-4a54-9621-a1996fadd8d3 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:18:44.640935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-18c69148-3350-4a54-9621-a1996fadd8d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:44.749829 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c69148-3350-4a54-9621-a1996fadd8d3 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:18:44.823901 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a99198-8a34-469e-816d-94053ab73bc0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:18:44.824416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a99198-8a34-469e-816d-94053ab73bc0 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:44.824690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a99198-8a34-469e-816d-94053ab73bc0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:44.824963 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a99198-8a34-469e-816d-94053ab73bc0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:44.826634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-26a99198-8a34-469e-816d-94053ab73bc0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:44.843850 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:44.940832 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a99198-8a34-469e-816d-94053ab73bc0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:18:44.942306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a99198-8a34-469e-816d-94053ab73bc0 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:18:44.960958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 665/1328] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:18:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:18:46.914031 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbc818-045d-4b94-9286-37a4596e07f4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:46.914607 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbc818-045d-4b94-9286-37a4596e07f4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:46.914722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbc818-045d-4b94-9286-37a4596e07f4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:46.916018 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 664/1329] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:18:46.921723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586085f0-8036-464f-837e-2c0e9ca8e5aa None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:46.922155 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586085f0-8036-464f-837e-2c0e9ca8e5aa None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:46.922449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586085f0-8036-464f-837e-2c0e9ca8e5aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:46.989386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-586085f0-8036-464f-837e-2c0e9ca8e5aa None None] MFA Rules not processed for user `179374916edf4dd480273f6c8c748ff6`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:18:46.990644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-586085f0-8036-464f-837e-2c0e9ca8e5aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:47.055879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:47.088404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 666/1330] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:18:46 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:18:47.373738 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbc818-045d-4b94-9286-37a4596e07f4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:18:47.375492 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-21dbc818-045d-4b94-9286-37a4596e07f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:47.489714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbc818-045d-4b94-9286-37a4596e07f4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:18:47.513484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:47.576145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffb945-21c7-46a6-9f3f-0b683f097d98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:18:47.577020 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb945-21c7-46a6-9f3f-0b683f097d98 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:47.577192 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb945-21c7-46a6-9f3f-0b683f097d98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:47.577589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb945-21c7-46a6-9f3f-0b683f097d98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:47.579270 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-74ffb945-21c7-46a6-9f3f-0b683f097d98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:47.702106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffb945-21c7-46a6-9f3f-0b683f097d98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:18:47.703587 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffb945-21c7-46a6-9f3f-0b683f097d98 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:18:47.721499 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 665/1331] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:18:47 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:18:49.982346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96763632-a78a-4799-b651-d33bae4ead8e None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:49.982892 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96763632-a78a-4799-b651-d33bae4ead8e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:49.982986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96763632-a78a-4799-b651-d33bae4ead8e None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:49.984867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 667/1332] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:18:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:18:49.990472 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b5cda-dc1b-461d-b3cd-013c9a863e1c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:49.990935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b5cda-dc1b-461d-b3cd-013c9a863e1c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:49.991029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771b5cda-dc1b-461d-b3cd-013c9a863e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:50.055596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:50.058501 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-771b5cda-dc1b-461d-b3cd-013c9a863e1c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:18:50.060944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-771b5cda-dc1b-461d-b3cd-013c9a863e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:50.160830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 666/1333] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:18:50.448313 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96763632-a78a-4799-b651-d33bae4ead8e None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:18:50.450396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-96763632-a78a-4799-b651-d33bae4ead8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:50.555480 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96763632-a78a-4799-b651-d33bae4ead8e None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:18:50.627209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fe2e7-0677-4fac-93fb-45ded854b129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:18:50.627807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fe2e7-0677-4fac-93fb-45ded854b129 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:50.628081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fe2e7-0677-4fac-93fb-45ded854b129 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:50.628351 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fe2e7-0677-4fac-93fb-45ded854b129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:50.629939 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-691fe2e7-0677-4fac-93fb-45ded854b129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:50.658393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:50.740302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fe2e7-0677-4fac-93fb-45ded854b129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:18:50.741631 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fe2e7-0677-4fac-93fb-45ded854b129 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:18:50.759662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 668/1334] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:18:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:18:52.704297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3add89-43d2-400a-aa00-6ec71c021dd0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:52.704835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3add89-43d2-400a-aa00-6ec71c021dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:52.704835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3add89-43d2-400a-aa00-6ec71c021dd0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:52.706185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 667/1335] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:18:52.711681 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a674b8-32b0-4c3a-9a59-0c27433ea6e8 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:52.712118 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a674b8-32b0-4c3a-9a59-0c27433ea6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:52.712213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a674b8-32b0-4c3a-9a59-0c27433ea6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:52.778365 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-60a674b8-32b0-4c3a-9a59-0c27433ea6e8 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:18:52.779594 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-60a674b8-32b0-4c3a-9a59-0c27433ea6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:52.851904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:52.875283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 669/1336] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:18:53.163061 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3add89-43d2-400a-aa00-6ec71c021dd0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:18:53.164624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dc3add89-43d2-400a-aa00-6ec71c021dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:53.268536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3add89-43d2-400a-aa00-6ec71c021dd0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:18:53.297876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:53.343098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3102ed74-49ef-4015-b9bc-9ddd9cac1752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:18:53.343705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102ed74-49ef-4015-b9bc-9ddd9cac1752 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:53.343986 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102ed74-49ef-4015-b9bc-9ddd9cac1752 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:53.344259 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3102ed74-49ef-4015-b9bc-9ddd9cac1752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:53.345855 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3102ed74-49ef-4015-b9bc-9ddd9cac1752 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:53.452975 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102ed74-49ef-4015-b9bc-9ddd9cac1752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:18:53.454434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3102ed74-49ef-4015-b9bc-9ddd9cac1752 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:18:53.472685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 668/1337] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:18:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:18:55.664735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15b5e1-18dc-48d4-a618-09ffeff6b7f1 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:55.665270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15b5e1-18dc-48d4-a618-09ffeff6b7f1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:55.665270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b15b5e1-18dc-48d4-a618-09ffeff6b7f1 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:55.666597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 670/1338] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:18:55.671807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eecaac-5474-4ba6-94b1-1d33f3f272f9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:55.672211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eecaac-5474-4ba6-94b1-1d33f3f272f9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:55.672351 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eecaac-5474-4ba6-94b1-1d33f3f272f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:55.738041 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-25eecaac-5474-4ba6-94b1-1d33f3f272f9 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:18:55.739263 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-25eecaac-5474-4ba6-94b1-1d33f3f272f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:55.747917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:55.836622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 669/1339] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:18:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:18:56.118787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15b5e1-18dc-48d4-a618-09ffeff6b7f1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:18:56.120312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8b15b5e1-18dc-48d4-a618-09ffeff6b7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:56.223985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b15b5e1-18dc-48d4-a618-09ffeff6b7f1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:18:56.296714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4df84d-d21f-4e44-b3dc-626eee2d5390 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:18:56.297307 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4df84d-d21f-4e44-b3dc-626eee2d5390 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:18:56.297580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4df84d-d21f-4e44-b3dc-626eee2d5390 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:18:56.297845 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4df84d-d21f-4e44-b3dc-626eee2d5390 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:18:56.299440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bc4df84d-d21f-4e44-b3dc-626eee2d5390 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:18:56.335575 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:18:56.409052 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4df84d-d21f-4e44-b3dc-626eee2d5390 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:18:56.410423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4df84d-d21f-4e44-b3dc-626eee2d5390 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:18:56.428826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 671/1340] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:18:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:19:00.299422 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2864d99-3273-4b5c-8ff2-dac119966fec None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:00.300105 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2864d99-3273-4b5c-8ff2-dac119966fec None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:00.300105 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2864d99-3273-4b5c-8ff2-dac119966fec None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:00.301316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 670/1341] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:00.306876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef13527-89b1-4d3f-81c4-597b803b7f22 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:00.307269 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef13527-89b1-4d3f-81c4-597b803b7f22 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:00.307466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef13527-89b1-4d3f-81c4-597b803b7f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:00.376280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5ef13527-89b1-4d3f-81c4-597b803b7f22 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:00.377413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5ef13527-89b1-4d3f-81c4-597b803b7f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:00.486934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:00.505519 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 672/1342] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:00.795786 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2864d99-3273-4b5c-8ff2-dac119966fec None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:00.797419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d2864d99-3273-4b5c-8ff2-dac119966fec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:00.924005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2864d99-3273-4b5c-8ff2-dac119966fec None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:00.969586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:01.007038 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa1e1ec-784c-4681-b115-835549db6ebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:01.007652 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1e1ec-784c-4681-b115-835549db6ebc service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:01.007927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1e1ec-784c-4681-b115-835549db6ebc service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:01.008192 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa1e1ec-784c-4681-b115-835549db6ebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:01.009839 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6aa1e1ec-784c-4681-b115-835549db6ebc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:01.139213 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1e1ec-784c-4681-b115-835549db6ebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:01.140556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa1e1ec-784c-4681-b115-835549db6ebc service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:01.160878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 671/1343] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:19:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:19:03.752403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc58f2-dde7-455f-a3a5-ba95c0d1874c None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:03.752996 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc58f2-dde7-455f-a3a5-ba95c0d1874c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:03.752996 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edc58f2-dde7-455f-a3a5-ba95c0d1874c None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:03.754299 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 673/1344] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:03.759560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a66a36-d073-4680-91f0-e8173d4315c4 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:03.760010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a66a36-d073-4680-91f0-e8173d4315c4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:03.760114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a66a36-d073-4680-91f0-e8173d4315c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:03.827820 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-75a66a36-d073-4680-91f0-e8173d4315c4 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:03.829038 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-75a66a36-d073-4680-91f0-e8173d4315c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:03.849389 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:03.932988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 672/1345] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:04.218462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc58f2-dde7-455f-a3a5-ba95c0d1874c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:04.220068 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6edc58f2-dde7-455f-a3a5-ba95c0d1874c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:04.330789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edc58f2-dde7-455f-a3a5-ba95c0d1874c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:04.410211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02d425-0620-448d-afc8-371f384a6193 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:04.410852 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02d425-0620-448d-afc8-371f384a6193 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:04.411134 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02d425-0620-448d-afc8-371f384a6193 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:04.411406 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02d425-0620-448d-afc8-371f384a6193 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:04.413009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3c02d425-0620-448d-afc8-371f384a6193 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:04.459638 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:04.540143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02d425-0620-448d-afc8-371f384a6193 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:04.541625 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02d425-0620-448d-afc8-371f384a6193 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:04.567800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 674/1346] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:19:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:19:08.471699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991de4b9-ce1c-4bef-8f86-f456a69c1924 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:08.472250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991de4b9-ce1c-4bef-8f86-f456a69c1924 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:08.472250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991de4b9-ce1c-4bef-8f86-f456a69c1924 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:08.473642 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 673/1347] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:08.479748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1229a93-038a-434a-8b06-f18ba8d850c2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:08.480295 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1229a93-038a-434a-8b06-f18ba8d850c2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:08.480700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1229a93-038a-434a-8b06-f18ba8d850c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:08.582863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e1229a93-038a-434a-8b06-f18ba8d850c2 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:08.584146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e1229a93-038a-434a-8b06-f18ba8d850c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:08.707438 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:08.710073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 675/1348] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:09.002479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991de4b9-ce1c-4bef-8f86-f456a69c1924 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:09.004175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-991de4b9-ce1c-4bef-8f86-f456a69c1924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:09.140017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991de4b9-ce1c-4bef-8f86-f456a69c1924 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:09.209523 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:09.236440 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4835a2f6-237e-4d09-aeea-5ca0a9603d99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:09.237033 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835a2f6-237e-4d09-aeea-5ca0a9603d99 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:09.237307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835a2f6-237e-4d09-aeea-5ca0a9603d99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:09.237577 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4835a2f6-237e-4d09-aeea-5ca0a9603d99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:09.239269 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4835a2f6-237e-4d09-aeea-5ca0a9603d99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:09.356706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835a2f6-237e-4d09-aeea-5ca0a9603d99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:09.358289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4835a2f6-237e-4d09-aeea-5ca0a9603d99 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:09.378670 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 674/1349] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:19:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:19:14.343547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21299bfb-f069-4e7a-8b72-5dbe757c5b3a None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:14.344084 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21299bfb-f069-4e7a-8b72-5dbe757c5b3a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:14.344563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21299bfb-f069-4e7a-8b72-5dbe757c5b3a None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:14.348373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 676/1350] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 15:19:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:14.355982 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d140fb-4f57-4b0b-8c72-3b87b1f1222a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:14.356432 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d140fb-4f57-4b0b-8c72-3b87b1f1222a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:14.356745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d140fb-4f57-4b0b-8c72-3b87b1f1222a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:14.455935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-90d140fb-4f57-4b0b-8c72-3b87b1f1222a None None] MFA Rules not processed for user `fb745b98e3ae4723bad44428646f83e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:14.457420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-90d140fb-4f57-4b0b-8c72-3b87b1f1222a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:14.481513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:14.614624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 675/1351] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 15:19:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:19.435212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28ad36-7f16-4714-81c4-79cc17cb7a58 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:19.435212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28ad36-7f16-4714-81c4-79cc17cb7a58 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:19.435212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a28ad36-7f16-4714-81c4-79cc17cb7a58 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:19.436293 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 677/1352] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 15:19:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:19.441191 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c91c9-152a-42d8-891a-0647f4abc63f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:19.441819 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c91c9-152a-42d8-891a-0647f4abc63f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:19.441819 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172c91c9-152a-42d8-891a-0647f4abc63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:19.558124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-172c91c9-152a-42d8-891a-0647f4abc63f None None] MFA Rules not processed for user `fb745b98e3ae4723bad44428646f83e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:19.559382 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-172c91c9-152a-42d8-891a-0647f4abc63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:19.657259 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:19.708491 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 676/1353] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 15:19:19 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:20.939597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc11db9-548b-4d38-8e8a-447ee558a481 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:20.940207 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc11db9-548b-4d38-8e8a-447ee558a481 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:20.940207 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc11db9-548b-4d38-8e8a-447ee558a481 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:20.941468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 678/1354] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 15:19:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:20.946570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f16429-06d9-402a-9acb-af67247f7204 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:20.946972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f16429-06d9-402a-9acb-af67247f7204 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:20.947090 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f16429-06d9-402a-9acb-af67247f7204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:21.046715 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-55f16429-06d9-402a-9acb-af67247f7204 None None] MFA Rules not processed for user `fb745b98e3ae4723bad44428646f83e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:21.048222 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-55f16429-06d9-402a-9acb-af67247f7204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:21.159609 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 677/1355] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 15:19:20 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:19:24.309934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6ee18-db1d-4bca-b1e8-54a5f31b06f8 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:24.310924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6ee18-db1d-4bca-b1e8-54a5f31b06f8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:24.311313 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a6ee18-db1d-4bca-b1e8-54a5f31b06f8 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:24.313203 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 679/1356] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:19:24.320719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb80f5-52d9-46a1-9046-987234c4a17b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:24.321254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb80f5-52d9-46a1-9046-987234c4a17b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:24.321379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcb80f5-52d9-46a1-9046-987234c4a17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:24.361020 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:24.409927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fdcb80f5-52d9-46a1-9046-987234c4a17b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:24.411384 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fdcb80f5-52d9-46a1-9046-987234c4a17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:24.552702 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 678/1357] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:24.889102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78679d1c-524b-4117-81d3-8bd6f2a8c171 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:24.889382 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78679d1c-524b-4117-81d3-8bd6f2a8c171 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:24.889653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78679d1c-524b-4117-81d3-8bd6f2a8c171 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:24.891453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 680/1358] 10.222.0.76 () {54 vars in 962 bytes} [Wed Aug 24 15:19:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:24.900082 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684a396-ed41-4303-a7a0-0d6e88919939 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:24.900082 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684a396-ed41-4303-a7a0-0d6e88919939 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:24.900874 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d684a396-ed41-4303-a7a0-0d6e88919939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:25.023129 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d684a396-ed41-4303-a7a0-0d6e88919939 None None] MFA Rules not processed for user `8a5fd31a2db944f596912e4f0aa26835`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:25.024620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d684a396-ed41-4303-a7a0-0d6e88919939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:25.095591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:25.193588 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 679/1359] 10.222.0.76 () {58 vars in 1059 bytes} [Wed Aug 24 15:19:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:25.206397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78679d1c-524b-4117-81d3-8bd6f2a8c171 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:25.215748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-78679d1c-524b-4117-81d3-8bd6f2a8c171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:25.388251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78679d1c-524b-4117-81d3-8bd6f2a8c171 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:25.491225 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:25.547443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce51c649-b624-4e3e-99a4-fb3e80102c0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:25.548653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51c649-b624-4e3e-99a4-fb3e80102c0c service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:25.548943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51c649-b624-4e3e-99a4-fb3e80102c0c service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:25.549216 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce51c649-b624-4e3e-99a4-fb3e80102c0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:25.551160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ce51c649-b624-4e3e-99a4-fb3e80102c0c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:25.669424 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51c649-b624-4e3e-99a4-fb3e80102c0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:25.671069 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce51c649-b624-4e3e-99a4-fb3e80102c0c service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:25.697848 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 681/1360] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:19:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:19:27.951575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6313bb-559d-46fa-9264-88b0e606e126 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:27.952428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6313bb-559d-46fa-9264-88b0e606e126 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:27.952428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6313bb-559d-46fa-9264-88b0e606e126 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:27.953652 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 680/1361] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:19:27.959195 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c03ec1-1a44-4a7d-93c7-7e8fcebbac30 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:27.959992 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c03ec1-1a44-4a7d-93c7-7e8fcebbac30 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:27.959992 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c03ec1-1a44-4a7d-93c7-7e8fcebbac30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:28.031253 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-44c03ec1-1a44-4a7d-93c7-7e8fcebbac30 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:28.032389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-44c03ec1-1a44-4a7d-93c7-7e8fcebbac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:28.080260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:28.160368 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 682/1362] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:19:28.462291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6313bb-559d-46fa-9264-88b0e606e126 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:28.464433 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0f6313bb-559d-46fa-9264-88b0e606e126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:28.654583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6313bb-559d-46fa-9264-88b0e606e126 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:28.678514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:28.820044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166d9f64-9202-4e09-b2fe-03c90d85689d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:28.821078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d9f64-9202-4e09-b2fe-03c90d85689d service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:28.821815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d9f64-9202-4e09-b2fe-03c90d85689d service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:28.822057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166d9f64-9202-4e09-b2fe-03c90d85689d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:28.824800 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-166d9f64-9202-4e09-b2fe-03c90d85689d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:28.959636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d9f64-9202-4e09-b2fe-03c90d85689d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:28.961340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166d9f64-9202-4e09-b2fe-03c90d85689d service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:28.984146 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 681/1363] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:19:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 522 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:19:31.321373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c56674-326a-49eb-a53d-9017e591a29b None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:31.321927 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c56674-326a-49eb-a53d-9017e591a29b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:31.322210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c56674-326a-49eb-a53d-9017e591a29b None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:31.324156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 683/1364] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:31.334139 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a46a7-daf5-406d-9f07-ab2deef2a170 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:31.334700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a46a7-daf5-406d-9f07-ab2deef2a170 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:31.334780 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a46a7-daf5-406d-9f07-ab2deef2a170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:31.352459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:31.435683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-419a46a7-daf5-406d-9f07-ab2deef2a170 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:31.436130 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-419a46a7-daf5-406d-9f07-ab2deef2a170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:31.545208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 682/1365] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:19:31.851440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eeeef5-ef1c-4c15-a799-abc5b27bf407 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:31.852358 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eeeef5-ef1c-4c15-a799-abc5b27bf407 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:31.852826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eeeef5-ef1c-4c15-a799-abc5b27bf407 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:31.855902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 684/1366] 10.222.0.76 () {54 vars in 962 bytes} [Wed Aug 24 15:19:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:31.864685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447bfdf-c5b1-4bfc-bc77-507788528502 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:31.865216 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447bfdf-c5b1-4bfc-bc77-507788528502 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:31.865558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447bfdf-c5b1-4bfc-bc77-507788528502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:31.961759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1447bfdf-c5b1-4bfc-bc77-507788528502 None None] MFA Rules not processed for user `8a5fd31a2db944f596912e4f0aa26835`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:31.961759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1447bfdf-c5b1-4bfc-bc77-507788528502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:32.002058 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:32.097030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 683/1367] 10.222.0.76 () {58 vars in 1059 bytes} [Wed Aug 24 15:19:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:32.106974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eeeef5-ef1c-4c15-a799-abc5b27bf407 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:32.108362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-76eeeef5-ef1c-4c15-a799-abc5b27bf407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:32.324707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eeeef5-ef1c-4c15-a799-abc5b27bf407 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:32.356536 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:32.482575 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101e307-3bb4-4559-bc27-7a107cdd5e52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:32.483914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101e307-3bb4-4559-bc27-7a107cdd5e52 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:32.484251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101e307-3bb4-4559-bc27-7a107cdd5e52 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:32.484881 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101e307-3bb4-4559-bc27-7a107cdd5e52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:32.487658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9101e307-3bb4-4559-bc27-7a107cdd5e52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:32.635060 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101e307-3bb4-4559-bc27-7a107cdd5e52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:32.636960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9101e307-3bb4-4559-bc27-7a107cdd5e52 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:32.667419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 685/1368] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:19:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:19:33.210104 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871f62b-e88f-4fcf-86c1-61412830900f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:33.210997 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871f62b-e88f-4fcf-86c1-61412830900f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:33.210997 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f871f62b-e88f-4fcf-86c1-61412830900f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:33.212346 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 684/1369] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:33.232593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b79b4-e0f7-4c89-85d3-0d1ae77b5542 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:33.233533 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b79b4-e0f7-4c89-85d3-0d1ae77b5542 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:33.234334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b79b4-e0f7-4c89-85d3-0d1ae77b5542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:33.293850 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:33.343951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-804b79b4-e0f7-4c89-85d3-0d1ae77b5542 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:33.345440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-804b79b4-e0f7-4c89-85d3-0d1ae77b5542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:33.472724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 686/1370] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:34.092605 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b7927-c4d1-4d37-8211-512ddfdf7600 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:34.093171 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b7927-c4d1-4d37-8211-512ddfdf7600 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:34.093603 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b7927-c4d1-4d37-8211-512ddfdf7600 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:34.095110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 685/1371] 10.222.0.76 () {54 vars in 964 bytes} [Wed Aug 24 15:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:34.100126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf2c63-5d77-4540-82d3-9843ede0f1c2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:34.100612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf2c63-5d77-4540-82d3-9843ede0f1c2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:34.101044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf2c63-5d77-4540-82d3-9843ede0f1c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:34.209405 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-1fdf2c63-5d77-4540-82d3-9843ede0f1c2 None None] MFA Rules not processed for user `194c7492cf954a6a8dd21de66aa24fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:34.210828 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1fdf2c63-5d77-4540-82d3-9843ede0f1c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:34.286833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:34.332334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 687/1372] 10.222.0.76 () {58 vars in 1061 bytes} [Wed Aug 24 15:19:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:34.340962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b7927-c4d1-4d37-8211-512ddfdf7600 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:34.343367 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dd4b7927-c4d1-4d37-8211-512ddfdf7600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:34.485310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b7927-c4d1-4d37-8211-512ddfdf7600 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:34.539461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:34.582491 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b6ce6c-e975-43fa-af66-03a749062139 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:34.583030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6ce6c-e975-43fa-af66-03a749062139 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:34.583309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6ce6c-e975-43fa-af66-03a749062139 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:34.583573 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b6ce6c-e975-43fa-af66-03a749062139 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:34.585384 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-43b6ce6c-e975-43fa-af66-03a749062139 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:34.741051 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6ce6c-e975-43fa-af66-03a749062139 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:34.742581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b6ce6c-e975-43fa-af66-03a749062139 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:34.766940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 686/1373] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:19:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:19:34.880103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dfbcb-6926-4d44-818b-eafa267b2153 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:34.880555 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dfbcb-6926-4d44-818b-eafa267b2153 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:34.880656 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01dfbcb-6926-4d44-818b-eafa267b2153 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:34.882125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 688/1374] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:34.888728 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af749f-e6ac-46a6-9509-417431523e67 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:34.889036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af749f-e6ac-46a6-9509-417431523e67 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:34.889342 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0af749f-e6ac-46a6-9509-417431523e67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:35.001685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f0af749f-e6ac-46a6-9509-417431523e67 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:35.003205 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f0af749f-e6ac-46a6-9509-417431523e67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:35.042696 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:35.162648 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 687/1375] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:35.466791 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dfbcb-6926-4d44-818b-eafa267b2153 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:35.469064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d01dfbcb-6926-4d44-818b-eafa267b2153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:35.654508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01dfbcb-6926-4d44-818b-eafa267b2153 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:35.717019 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:35.787505 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8237703-5f20-4ff1-b793-53ebf51d7171 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:35.788118 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8237703-5f20-4ff1-b793-53ebf51d7171 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:35.788394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8237703-5f20-4ff1-b793-53ebf51d7171 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:35.788657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8237703-5f20-4ff1-b793-53ebf51d7171 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:35.790318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a8237703-5f20-4ff1-b793-53ebf51d7171 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:35.927629 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8237703-5f20-4ff1-b793-53ebf51d7171 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:35.929318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8237703-5f20-4ff1-b793-53ebf51d7171 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:35.955144 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 689/1376] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:19:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:19:38.024145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e32564-d36b-4c99-a895-10466c825842 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:38.024683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e32564-d36b-4c99-a895-10466c825842 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:38.024804 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e32564-d36b-4c99-a895-10466c825842 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:38.027995 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 688/1377] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:38.038325 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01e9ad-6858-42f5-9f60-bc1aada8e5f4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:38.038325 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01e9ad-6858-42f5-9f60-bc1aada8e5f4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:38.038325 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01e9ad-6858-42f5-9f60-bc1aada8e5f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:38.113329 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2b01e9ad-6858-42f5-9f60-bc1aada8e5f4 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:38.114518 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2b01e9ad-6858-42f5-9f60-bc1aada8e5f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:38.121721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:38.230147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 690/1378] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:19:38.551591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e32564-d36b-4c99-a895-10466c825842 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:38.553759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-50e32564-d36b-4c99-a895-10466c825842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:38.765810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e32564-d36b-4c99-a895-10466c825842 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:38.770385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:38.975590 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc295d1-d69c-48f7-8c38-4edde0390ce8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:38.976207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc295d1-d69c-48f7-8c38-4edde0390ce8 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:38.976488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc295d1-d69c-48f7-8c38-4edde0390ce8 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:38.976756 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc295d1-d69c-48f7-8c38-4edde0390ce8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:38.979611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-adc295d1-d69c-48f7-8c38-4edde0390ce8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:39.098385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc295d1-d69c-48f7-8c38-4edde0390ce8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:39.099936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc295d1-d69c-48f7-8c38-4edde0390ce8 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:39.121569 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 689/1379] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:19:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:19:41.270917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692ea16-faec-4c90-8f31-97c2de723b80 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:41.271649 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692ea16-faec-4c90-8f31-97c2de723b80 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:41.271649 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f692ea16-faec-4c90-8f31-97c2de723b80 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:41.272696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 691/1380] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:41.279224 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bab60-b740-417d-83f9-80e7271eea40 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:41.279697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bab60-b740-417d-83f9-80e7271eea40 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:41.279794 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75bab60-b740-417d-83f9-80e7271eea40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:41.291627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:41.357644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e75bab60-b740-417d-83f9-80e7271eea40 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:41.358899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e75bab60-b740-417d-83f9-80e7271eea40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:41.469264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 690/1381] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:41.764354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692ea16-faec-4c90-8f31-97c2de723b80 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:41.765912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f692ea16-faec-4c90-8f31-97c2de723b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:41.918801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:41.931494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f692ea16-faec-4c90-8f31-97c2de723b80 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:42.077804 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681452d7-c7ed-4756-8981-99fcecc70196 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:42.078463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681452d7-c7ed-4756-8981-99fcecc70196 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:42.078786 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681452d7-c7ed-4756-8981-99fcecc70196 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:42.079061 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681452d7-c7ed-4756-8981-99fcecc70196 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:42.080787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-681452d7-c7ed-4756-8981-99fcecc70196 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:42.191394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681452d7-c7ed-4756-8981-99fcecc70196 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:42.193437 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681452d7-c7ed-4756-8981-99fcecc70196 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:42.209020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:42.225508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 692/1382] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:19:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:19:44.296686 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b736b-dfc6-482c-b45a-152176fa6524 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:44.297374 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b736b-dfc6-482c-b45a-152176fa6524 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:44.297374 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1b736b-dfc6-482c-b45a-152176fa6524 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:44.299155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 691/1383] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:44.305345 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f2102-6404-468b-a7d0-dc9f74f6e89d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:44.305926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f2102-6404-468b-a7d0-dc9f74f6e89d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:44.306026 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f2102-6404-468b-a7d0-dc9f74f6e89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:44.388004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e91f2102-6404-468b-a7d0-dc9f74f6e89d None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:44.389484 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e91f2102-6404-468b-a7d0-dc9f74f6e89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:44.494698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 693/1384] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:44.786572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b736b-dfc6-482c-b45a-152176fa6524 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:44.788200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1a1b736b-dfc6-482c-b45a-152176fa6524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:44.890382 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:44.976053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1b736b-dfc6-482c-b45a-152176fa6524 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:45.156711 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f38ce2-de91-44d2-9a67-3d2d582b5b1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:45.157331 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f38ce2-de91-44d2-9a67-3d2d582b5b1a service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:45.157616 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f38ce2-de91-44d2-9a67-3d2d582b5b1a service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:45.157893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f38ce2-de91-44d2-9a67-3d2d582b5b1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:45.159734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-09f38ce2-de91-44d2-9a67-3d2d582b5b1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:45.225942 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:45.295405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f38ce2-de91-44d2-9a67-3d2d582b5b1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:45.296956 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f38ce2-de91-44d2-9a67-3d2d582b5b1a service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:45.318270 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 692/1385] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:19:47.345707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce628db-8294-474b-ba25-e28dc1a98954 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:47.346510 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce628db-8294-474b-ba25-e28dc1a98954 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:47.346510 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce628db-8294-474b-ba25-e28dc1a98954 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:47.347761 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 694/1386] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:47.353112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57abc43-6e2b-467c-a9e3-cc806bdf5493 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:47.353632 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57abc43-6e2b-467c-a9e3-cc806bdf5493 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:47.353731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57abc43-6e2b-467c-a9e3-cc806bdf5493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:47.446789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a57abc43-6e2b-467c-a9e3-cc806bdf5493 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:47.450314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a57abc43-6e2b-467c-a9e3-cc806bdf5493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:47.604122 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 693/1387] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:47.896910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce628db-8294-474b-ba25-e28dc1a98954 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:47.898486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6ce628db-8294-474b-ba25-e28dc1a98954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:48.002703 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:48.080217 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce628db-8294-474b-ba25-e28dc1a98954 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:48.233844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec6bf2-7192-4e49-9f2e-23cfbbb5c1f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:48.234708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec6bf2-7192-4e49-9f2e-23cfbbb5c1f8 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:48.235099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec6bf2-7192-4e49-9f2e-23cfbbb5c1f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:48.235540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec6bf2-7192-4e49-9f2e-23cfbbb5c1f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:48.237526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f5ec6bf2-7192-4e49-9f2e-23cfbbb5c1f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:48.289027 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:48.349515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec6bf2-7192-4e49-9f2e-23cfbbb5c1f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:48.351391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec6bf2-7192-4e49-9f2e-23cfbbb5c1f8 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:48.373803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 695/1388] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:19:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:19:50.485034 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e6dbd-c739-46b0-a605-8e180e343d2d None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:50.485927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e6dbd-c739-46b0-a605-8e180e343d2d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:50.485927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6e6dbd-c739-46b0-a605-8e180e343d2d None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:50.487096 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 694/1389] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:50.493980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79a69f-fd1d-49a9-8b1a-3935cf01d4c6 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:50.495380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79a69f-fd1d-49a9-8b1a-3935cf01d4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:50.495585 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79a69f-fd1d-49a9-8b1a-3935cf01d4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:50.602403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cf79a69f-fd1d-49a9-8b1a-3935cf01d4c6 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:50.603880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cf79a69f-fd1d-49a9-8b1a-3935cf01d4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:50.732509 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 696/1390] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:50.824227 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155cce6-42e0-46ee-9dc4-c73aa6fa2091 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:50.824455 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155cce6-42e0-46ee-9dc4-c73aa6fa2091 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:50.824664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155cce6-42e0-46ee-9dc4-c73aa6fa2091 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:50.826048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 695/1391] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:19:50.832102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525363f-b60b-49e3-86f9-8cb973a44a2f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:50.832880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525363f-b60b-49e3-86f9-8cb973a44a2f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:50.833157 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525363f-b60b-49e3-86f9-8cb973a44a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:50.853247 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:50.926992 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b525363f-b60b-49e3-86f9-8cb973a44a2f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:50.928238 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b525363f-b60b-49e3-86f9-8cb973a44a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:51.069795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155cce6-42e0-46ee-9dc4-c73aa6fa2091 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:51.071785 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9155cce6-42e0-46ee-9dc4-c73aa6fa2091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:51.075055 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 697/1392] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:51.122319 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:51.272902 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155cce6-42e0-46ee-9dc4-c73aa6fa2091 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:51.419639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:51.441101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d3d2b-3625-45fb-9401-f0cd91969341 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:51.441717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d3d2b-3625-45fb-9401-f0cd91969341 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:51.441997 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d3d2b-3625-45fb-9401-f0cd91969341 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:51.442294 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d3d2b-3625-45fb-9401-f0cd91969341 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:51.443975 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2f7d3d2b-3625-45fb-9401-f0cd91969341 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:51.462883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a58ae-0607-4a43-a978-e95721c4f230 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:51.463259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a58ae-0607-4a43-a978-e95721c4f230 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:51.463536 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2a58ae-0607-4a43-a978-e95721c4f230 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:51.464908 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 698/1393] 10.222.0.76 () {54 vars in 967 bytes} [Wed Aug 24 15:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:19:51.471343 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3ff1b-b3be-4db3-9842-41e9bbab435c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:51.471625 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3ff1b-b3be-4db3-9842-41e9bbab435c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:51.471895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3ff1b-b3be-4db3-9842-41e9bbab435c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:51.549654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d3d2b-3625-45fb-9401-f0cd91969341 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:51.551386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d3d2b-3625-45fb-9401-f0cd91969341 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:51.565370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-33d3ff1b-b3be-4db3-9842-41e9bbab435c None None] MFA Rules not processed for user `96c71ea88d9a474c9ff7a30555cfa91f`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:51.566880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-33d3ff1b-b3be-4db3-9842-41e9bbab435c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:51.570980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 696/1394] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:19:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:19:51.609980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:51.668902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 699/1395] 10.222.0.76 () {58 vars in 1064 bytes} [Wed Aug 24 15:19:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:19:51.677275 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d3d2b-3625-45fb-9401-f0cd91969341 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:51.678612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2f7d3d2b-3625-45fb-9401-f0cd91969341 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:51.778113 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d3d2b-3625-45fb-9401-f0cd91969341 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:51.835029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:51.874545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd5b94c-00ed-4ec1-8f11-b90930afd639 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96c71ea88d9a474c9ff7a30555cfa91f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:51.875195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5b94c-00ed-4ec1-8f11-b90930afd639 service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:51.875479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5b94c-00ed-4ec1-8f11-b90930afd639 service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:51.875751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd5b94c-00ed-4ec1-8f11-b90930afd639 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:51.877490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4fd5b94c-00ed-4ec1-8f11-b90930afd639 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:52.018945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd5b94c-00ed-4ec1-8f11-b90930afd639 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:52.020505 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd5b94c-00ed-4ec1-8f11-b90930afd639 service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:52.043223 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 697/1396] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:19:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:19:52.810954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e19418-fd18-471c-83a8-199de37a7160 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:52.811477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e19418-fd18-471c-83a8-199de37a7160 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:52.811477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e19418-fd18-471c-83a8-199de37a7160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:52.891407 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a6e19418-fd18-471c-83a8-199de37a7160 None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:52.892614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a6e19418-fd18-471c-83a8-199de37a7160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:52.994920 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 700/1397] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 15:19:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:53.040322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7f903-5010-4b56-a4fc-9d36d5555891 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:53.041201 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7f903-5010-4b56-a4fc-9d36d5555891 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:53.041201 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a7f903-5010-4b56-a4fc-9d36d5555891 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:53.042365 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 698/1398] 10.222.0.76 () {54 vars in 943 bytes} [Wed Aug 24 15:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:53.048917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf80b34-293f-4faa-ac81-ded8eeb3449d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:53.049211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf80b34-293f-4faa-ac81-ded8eeb3449d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:53.049472 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf80b34-293f-4faa-ac81-ded8eeb3449d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:53.113746 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:53.145478 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6bf80b34-293f-4faa-ac81-ded8eeb3449d None None] MFA Rules not processed for user `0679072b37db41e880a95b738bea47ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:53.146810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6bf80b34-293f-4faa-ac81-ded8eeb3449d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:53.287956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 701/1399] 10.222.0.76 () {58 vars in 1040 bytes} [Wed Aug 24 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:53.298383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7f903-5010-4b56-a4fc-9d36d5555891 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:53.300223 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d8a7f903-5010-4b56-a4fc-9d36d5555891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:53.394372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:53.464163 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a7f903-5010-4b56-a4fc-9d36d5555891 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:53.591903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264e762-fbc2-4dca-9072-5b754e2d618c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:53.592437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264e762-fbc2-4dca-9072-5b754e2d618c service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:53.592719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264e762-fbc2-4dca-9072-5b754e2d618c service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:53.592994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d264e762-fbc2-4dca-9072-5b754e2d618c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:53.595147 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d264e762-fbc2-4dca-9072-5b754e2d618c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:53.606206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f516a-d44c-49bc-a2b6-7852e8fc7cac None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:53.606526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f516a-d44c-49bc-a2b6-7852e8fc7cac None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:53.606800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1f516a-d44c-49bc-a2b6-7852e8fc7cac None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:53.608165 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 702/1400] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:19:53.614270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d195a1e-8190-456b-9b03-f41f1b903483 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:53.614566 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d195a1e-8190-456b-9b03-f41f1b903483 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:53.614843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d195a1e-8190-456b-9b03-f41f1b903483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:53.663985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:53.688830 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8d195a1e-8190-456b-9b03-f41f1b903483 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:53.690095 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8d195a1e-8190-456b-9b03-f41f1b903483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:53.694765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d264e762-fbc2-4dca-9072-5b754e2d618c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:53.696392 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d264e762-fbc2-4dca-9072-5b754e2d618c service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:53.707407 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 699/1401] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:19:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:19:53.786884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:53.862046 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 703/1402] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:19:54.158812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264e762-fbc2-4dca-9072-5b754e2d618c service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:54.160189 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d264e762-fbc2-4dca-9072-5b754e2d618c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:54.264756 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d264e762-fbc2-4dca-9072-5b754e2d618c service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:54.379224 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:54.392296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53e7735-1762-47df-8971-90238bc98fc5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:54.392950 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e7735-1762-47df-8971-90238bc98fc5 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:54.393180 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e7735-1762-47df-8971-90238bc98fc5 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:54.393449 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53e7735-1762-47df-8971-90238bc98fc5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:54.395111 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b53e7735-1762-47df-8971-90238bc98fc5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:54.509118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e7735-1762-47df-8971-90238bc98fc5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:54.510963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53e7735-1762-47df-8971-90238bc98fc5 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:54.538537 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 700/1403] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:19:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:19:56.576197 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80798f-73a4-458e-9126-f95f128582ee None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:56.576754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80798f-73a4-458e-9126-f95f128582ee None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:56.576939 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c80798f-73a4-458e-9126-f95f128582ee None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:56.582193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 704/1404] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:19:56.583918 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352353b-53db-4440-a41c-bcb97db173ea None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:56.584330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352353b-53db-4440-a41c-bcb97db173ea None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:56.584462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3352353b-53db-4440-a41c-bcb97db173ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:56.661412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3352353b-53db-4440-a41c-bcb97db173ea None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:56.662748 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3352353b-53db-4440-a41c-bcb97db173ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:56.735185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:56.774575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 701/1405] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:19:57.063793 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80798f-73a4-458e-9126-f95f128582ee None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:19:57.066257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3c80798f-73a4-458e-9126-f95f128582ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:57.256070 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c80798f-73a4-458e-9126-f95f128582ee None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:19:57.307686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:57.394545 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4543726-8edf-4902-8bdf-50a291d3b19c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:19:57.395147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4543726-8edf-4902-8bdf-50a291d3b19c service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:57.395421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4543726-8edf-4902-8bdf-50a291d3b19c service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:57.395684 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4543726-8edf-4902-8bdf-50a291d3b19c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:57.397263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b4543726-8edf-4902-8bdf-50a291d3b19c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:57.513272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4543726-8edf-4902-8bdf-50a291d3b19c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:19:57.514934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4543726-8edf-4902-8bdf-50a291d3b19c service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:19:57.539279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 705/1406] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:19:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:19:59.592841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf7c7d-d6aa-4108-b89f-1d701f13e19f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:59.593450 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf7c7d-d6aa-4108-b89f-1d701f13e19f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:59.593450 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddf7c7d-d6aa-4108-b89f-1d701f13e19f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:59.595226 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 702/1407] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:19:59.605065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d2d8e-e4d1-4e80-a030-4b44cf49c30e None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:19:59.605065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d2d8e-e4d1-4e80-a030-4b44cf49c30e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:19:59.605065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d2d8e-e4d1-4e80-a030-4b44cf49c30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:19:59.683538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:19:59.695551 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7b6d2d8e-e4d1-4e80-a030-4b44cf49c30e None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:19:59.697378 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7b6d2d8e-e4d1-4e80-a030-4b44cf49c30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:19:59.834609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 706/1408] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:19:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:00.158662 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf7c7d-d6aa-4108-b89f-1d701f13e19f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:00.163308 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3ddf7c7d-d6aa-4108-b89f-1d701f13e19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:00.364273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddf7c7d-d6aa-4108-b89f-1d701f13e19f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:00.417735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:00.521727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bed9c2a-ad60-4dfc-8e44-684f23b9b88f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:00.522387 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed9c2a-ad60-4dfc-8e44-684f23b9b88f service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:00.522675 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed9c2a-ad60-4dfc-8e44-684f23b9b88f service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:00.523048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bed9c2a-ad60-4dfc-8e44-684f23b9b88f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:00.524855 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0bed9c2a-ad60-4dfc-8e44-684f23b9b88f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:00.677689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed9c2a-ad60-4dfc-8e44-684f23b9b88f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:00.680066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bed9c2a-ad60-4dfc-8e44-684f23b9b88f service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:00.708171 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 703/1409] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:20:09.832943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=44 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:10.705927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=77 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:16.530102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535852f-d277-4587-aba8-bc55d7a8c0bb None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:16.530700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535852f-d277-4587-aba8-bc55d7a8c0bb None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:16.530820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e535852f-d277-4587-aba8-bc55d7a8c0bb None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:16.532423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 707/1410] 10.222.0.76 () {54 vars in 918 bytes} [Wed Aug 24 15:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:16.537907 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daf2da-b924-4764-89b9-1982277db550 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:16.538443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daf2da-b924-4764-89b9-1982277db550 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:16.538550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98daf2da-b924-4764-89b9-1982277db550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:16.621540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-98daf2da-b924-4764-89b9-1982277db550 None None] MFA Rules not processed for user `fb745b98e3ae4723bad44428646f83e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:16.622903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-98daf2da-b924-4764-89b9-1982277db550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:16.737024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 704/1411] 10.222.0.76 () {58 vars in 1015 bytes} [Wed Aug 24 15:20:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:16.752630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e775f6c-6aae-4189-8d90-802932c58cc0 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:16.752917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e775f6c-6aae-4189-8d90-802932c58cc0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:16.753187 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e775f6c-6aae-4189-8d90-802932c58cc0 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:16.754546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 708/1412] 10.222.0.76 () {54 vars in 918 bytes} [Wed Aug 24 15:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:16.759536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622bbc1-7b20-4bb7-9f1b-0d545defc574 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:16.759827 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622bbc1-7b20-4bb7-9f1b-0d545defc574 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:16.760135 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622bbc1-7b20-4bb7-9f1b-0d545defc574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:16.842357 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:16.845540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6622bbc1-7b20-4bb7-9f1b-0d545defc574 None None] MFA Rules not processed for user `8a5fd31a2db944f596912e4f0aa26835`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:16.847403 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6622bbc1-7b20-4bb7-9f1b-0d545defc574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:16.998154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 705/1413] 10.222.0.76 () {58 vars in 1015 bytes} [Wed Aug 24 15:20:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:20:19.399644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad5266-98af-4213-a230-ce62cfaebe98 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:19.400062 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad5266-98af-4213-a230-ce62cfaebe98 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:19.400193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad5266-98af-4213-a230-ce62cfaebe98 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:19.401822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:19.407610 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4acd58-fa51-426e-9d6d-1d6dd63bd23c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:19.408067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4acd58-fa51-426e-9d6d-1d6dd63bd23c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:19.408175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4acd58-fa51-426e-9d6d-1d6dd63bd23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:19.474066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0c4acd58-fa51-426e-9d6d-1d6dd63bd23c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:19.475314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0c4acd58-fa51-426e-9d6d-1d6dd63bd23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:19.532917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:19.576842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:20:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:19.866382 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad5266-98af-4213-a230-ce62cfaebe98 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:19.868070 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2bad5266-98af-4213-a230-ce62cfaebe98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:20.025938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad5266-98af-4213-a230-ce62cfaebe98 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:20.096951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:20.136843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d081fd-5a5f-4c47-bbc1-7888b786a14e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:20.137368 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d081fd-5a5f-4c47-bbc1-7888b786a14e service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:20.137642 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d081fd-5a5f-4c47-bbc1-7888b786a14e service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:20.137908 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d081fd-5a5f-4c47-bbc1-7888b786a14e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:20.139544 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-19d081fd-5a5f-4c47-bbc1-7888b786a14e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:20.230292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d081fd-5a5f-4c47-bbc1-7888b786a14e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:20.231914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d081fd-5a5f-4c47-bbc1-7888b786a14e service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:20.250068 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 710/1416] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:21.148906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6c4ed-220a-4ac5-8fcf-8a7f9fa686d7 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:21.149588 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6c4ed-220a-4ac5-8fcf-8a7f9fa686d7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:21.149588 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6c4ed-220a-4ac5-8fcf-8a7f9fa686d7 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:21.150931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:21.157041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ae1d3-d499-49ec-b48b-117ac2599243 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:21.157713 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ae1d3-d499-49ec-b48b-117ac2599243 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:21.157713 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ae1d3-d499-49ec-b48b-117ac2599243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:21.242395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6f0ae1d3-d499-49ec-b48b-117ac2599243 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:21.244208 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6f0ae1d3-d499-49ec-b48b-117ac2599243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:21.270400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:21.342098 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:20:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:21.494047 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6c4ed-220a-4ac5-8fcf-8a7f9fa686d7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:21.495621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f4c6c4ed-220a-4ac5-8fcf-8a7f9fa686d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:21.642668 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6c4ed-220a-4ac5-8fcf-8a7f9fa686d7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:21.684297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:21.763734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c2c12a-94bc-47b3-a395-a5c19c55068f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:21.764340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2c12a-94bc-47b3-a395-a5c19c55068f service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:21.764621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2c12a-94bc-47b3-a395-a5c19c55068f service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:21.764893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c2c12a-94bc-47b3-a395-a5c19c55068f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:21.766543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b7c2c12a-94bc-47b3-a395-a5c19c55068f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:21.855679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2c12a-94bc-47b3-a395-a5c19c55068f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:21.857273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c2c12a-94bc-47b3-a395-a5c19c55068f service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:21.875080 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 708/1419] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:22.752768 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb47f11-5302-4a10-8f26-41b97b5640a7 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:22.753417 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb47f11-5302-4a10-8f26-41b97b5640a7 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:22.753417 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb47f11-5302-4a10-8f26-41b97b5640a7 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:22.754817 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:22.760341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab1af5-9ca9-4567-96e5-5c2dac0e3653 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:22.760790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab1af5-9ca9-4567-96e5-5c2dac0e3653 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:22.760923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab1af5-9ca9-4567-96e5-5c2dac0e3653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:22.816626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:22.839758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-16ab1af5-9ca9-4567-96e5-5c2dac0e3653 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:22.840970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-16ab1af5-9ca9-4567-96e5-5c2dac0e3653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:22.936090 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:20:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:23.089383 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb47f11-5302-4a10-8f26-41b97b5640a7 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:23.090858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1eb47f11-5302-4a10-8f26-41b97b5640a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:23.236294 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb47f11-5302-4a10-8f26-41b97b5640a7 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:23.249274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:23.349107 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d11fa0-bc7f-4a29-bd25-fdc13c87cbdc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:23.349602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d11fa0-bc7f-4a29-bd25-fdc13c87cbdc service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:23.349880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d11fa0-bc7f-4a29-bd25-fdc13c87cbdc service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:23.350150 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d11fa0-bc7f-4a29-bd25-fdc13c87cbdc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:23.351775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d4d11fa0-bc7f-4a29-bd25-fdc13c87cbdc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:23.444153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d11fa0-bc7f-4a29-bd25-fdc13c87cbdc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:23.445786 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d11fa0-bc7f-4a29-bd25-fdc13c87cbdc service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:23.463831 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 713/1422] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:24.326743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813d608-3d78-4e5b-b33a-2736263911df None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:24.327315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813d608-3d78-4e5b-b33a-2736263911df None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:24.327315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813d608-3d78-4e5b-b33a-2736263911df None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:24.328631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:20:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:24.333501 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0dabc-cfc8-494c-beb6-a610720e9e7b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:24.333935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0dabc-cfc8-494c-beb6-a610720e9e7b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:24.334040 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed0dabc-cfc8-494c-beb6-a610720e9e7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:24.354564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:24.414003 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9ed0dabc-cfc8-494c-beb6-a610720e9e7b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:24.415904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9ed0dabc-cfc8-494c-beb6-a610720e9e7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:24.513297 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:20:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:24.663637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813d608-3d78-4e5b-b33a-2736263911df None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:24.665138 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2813d608-3d78-4e5b-b33a-2736263911df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:24.778157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:24.814658 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813d608-3d78-4e5b-b33a-2736263911df None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:24.933546 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a453bc-8d8d-463a-a78e-65b9b95ec4bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:24.934249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a453bc-8d8d-463a-a78e-65b9b95ec4bb service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:24.934526 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a453bc-8d8d-463a-a78e-65b9b95ec4bb service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:24.934971 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a453bc-8d8d-463a-a78e-65b9b95ec4bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:24.936396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-72a453bc-8d8d-463a-a78e-65b9b95ec4bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:25.014612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:25.029495 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a453bc-8d8d-463a-a78e-65b9b95ec4bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:25.031129 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a453bc-8d8d-463a-a78e-65b9b95ec4bb service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:25.049411 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 711/1425] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:25.925490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a350ec-5a20-42c1-82aa-6aaf4226f189 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:25.926050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a350ec-5a20-42c1-82aa-6aaf4226f189 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:25.926050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a350ec-5a20-42c1-82aa-6aaf4226f189 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:25.927414 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:20:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:25.932481 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee662a0e-1dbb-49ab-9c19-4fad885aae60 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:25.932887 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee662a0e-1dbb-49ab-9c19-4fad885aae60 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:25.933010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee662a0e-1dbb-49ab-9c19-4fad885aae60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:26.009620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ee662a0e-1dbb-49ab-9c19-4fad885aae60 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:26.010872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ee662a0e-1dbb-49ab-9c19-4fad885aae60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:26.106767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:26.259870 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a350ec-5a20-42c1-82aa-6aaf4226f189 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:26.261341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-85a350ec-5a20-42c1-82aa-6aaf4226f189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:26.345294 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:26.408219 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a350ec-5a20-42c1-82aa-6aaf4226f189 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:26.520836 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230a0313-acbb-4f3f-8d46-901a04f14a59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:26.521316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a0313-acbb-4f3f-8d46-901a04f14a59 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:26.521590 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a0313-acbb-4f3f-8d46-901a04f14a59 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:26.521860 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230a0313-acbb-4f3f-8d46-901a04f14a59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:26.523495 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-230a0313-acbb-4f3f-8d46-901a04f14a59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:26.576226 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:26.619818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a0313-acbb-4f3f-8d46-901a04f14a59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:26.620780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230a0313-acbb-4f3f-8d46-901a04f14a59 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:26.641681 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 716/1428] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:27.445546 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dea3c-ad6c-415f-a687-3339b2c9dd77 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:27.446046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dea3c-ad6c-415f-a687-3339b2c9dd77 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:27.446150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dea3c-ad6c-415f-a687-3339b2c9dd77 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:27.447498 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:20:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:20:27.452628 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c36c31-6877-463a-b3cc-16d45a15482b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:27.453102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c36c31-6877-463a-b3cc-16d45a15482b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:27.453202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c36c31-6877-463a-b3cc-16d45a15482b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:27.532887 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e9c36c31-6877-463a-b3cc-16d45a15482b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:27.534132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e9c36c31-6877-463a-b3cc-16d45a15482b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:27.633195 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:20:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:27.786420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172dea3c-ad6c-415f-a687-3339b2c9dd77 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:27.787946 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-172dea3c-ad6c-415f-a687-3339b2c9dd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:27.835894 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:27.934094 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172dea3c-ad6c-415f-a687-3339b2c9dd77 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:28.046742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02544043-0e61-4b3d-ba3f-2d5530c85607 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:28.047261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02544043-0e61-4b3d-ba3f-2d5530c85607 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:28.047536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02544043-0e61-4b3d-ba3f-2d5530c85607 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:28.047805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02544043-0e61-4b3d-ba3f-2d5530c85607 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:28.049408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-02544043-0e61-4b3d-ba3f-2d5530c85607 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:28.056330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:28.157029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02544043-0e61-4b3d-ba3f-2d5530c85607 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:28.158658 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02544043-0e61-4b3d-ba3f-2d5530c85607 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:28.185092 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 714/1431] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:28.989474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fbbee-ae33-41e0-ae55-f5f972b935d8 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:28.990025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fbbee-ae33-41e0-ae55-f5f972b935d8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:28.990025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67fbbee-ae33-41e0-ae55-f5f972b935d8 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:28.991409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:28.997032 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98111ce8-0bc1-40ce-bb37-7c2d03b216bd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:28.997451 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98111ce8-0bc1-40ce-bb37-7c2d03b216bd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:28.997605 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98111ce8-0bc1-40ce-bb37-7c2d03b216bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:29.073466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-98111ce8-0bc1-40ce-bb37-7c2d03b216bd None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:29.074733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-98111ce8-0bc1-40ce-bb37-7c2d03b216bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:29.145116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:29.174435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:20:29.326057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fbbee-ae33-41e0-ae55-f5f972b935d8 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:29.327534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e67fbbee-ae33-41e0-ae55-f5f972b935d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:29.345380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:29.474622 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67fbbee-ae33-41e0-ae55-f5f972b935d8 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:29.566485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:29.594487 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cef1fcb-cf0b-41f9-a11e-8b551410bbb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:29.595018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef1fcb-cf0b-41f9-a11e-8b551410bbb8 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:29.595292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef1fcb-cf0b-41f9-a11e-8b551410bbb8 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:29.595563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cef1fcb-cf0b-41f9-a11e-8b551410bbb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:29.597164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1cef1fcb-cf0b-41f9-a11e-8b551410bbb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:29.688209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef1fcb-cf0b-41f9-a11e-8b551410bbb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:29.689824 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cef1fcb-cf0b-41f9-a11e-8b551410bbb8 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:29.708289 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 719/1434] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:30.513081 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc92a0-5cb3-495c-9a23-4e80126d7d66 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:30.513628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc92a0-5cb3-495c-9a23-4e80126d7d66 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:30.513628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc92a0-5cb3-495c-9a23-4e80126d7d66 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:30.515025 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:30.520087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdecd6-9b12-4a90-a02d-3071c89faf88 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:30.520522 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdecd6-9b12-4a90-a02d-3071c89faf88 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:30.520661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fdecd6-9b12-4a90-a02d-3071c89faf88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:30.597356 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-91fdecd6-9b12-4a90-a02d-3071c89faf88 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:30.598640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-91fdecd6-9b12-4a90-a02d-3071c89faf88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:30.639593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:30.696567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:20:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:30.846887 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc92a0-5cb3-495c-9a23-4e80126d7d66 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:30.848513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-08bc92a0-5cb3-495c-9a23-4e80126d7d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:30.993982 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc92a0-5cb3-495c-9a23-4e80126d7d66 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:31.049561 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:31.110085 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dba09e2-f291-40c8-b9a9-cb54b45e5e5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:31.110643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba09e2-f291-40c8-b9a9-cb54b45e5e5b service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:31.110924 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba09e2-f291-40c8-b9a9-cb54b45e5e5b service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:31.111196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dba09e2-f291-40c8-b9a9-cb54b45e5e5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:31.112825 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4dba09e2-f291-40c8-b9a9-cb54b45e5e5b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:31.204654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba09e2-f291-40c8-b9a9-cb54b45e5e5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:31.206287 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dba09e2-f291-40c8-b9a9-cb54b45e5e5b service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:31.224161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 717/1437] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:32.028600 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a623c-9716-4fa9-8f06-492842ed1cde None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:32.029177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a623c-9716-4fa9-8f06-492842ed1cde None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:32.029177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8a623c-9716-4fa9-8f06-492842ed1cde None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:32.030550 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:32.039236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4ca5d-ce30-4c80-b283-d851a37f4a19 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:32.039681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4ca5d-ce30-4c80-b283-d851a37f4a19 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:32.039778 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b4ca5d-ce30-4c80-b283-d851a37f4a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:32.115353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d0b4ca5d-ce30-4c80-b283-d851a37f4a19 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:32.116578 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d0b4ca5d-ce30-4c80-b283-d851a37f4a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:32.123815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:32.214495 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:32.365586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a623c-9716-4fa9-8f06-492842ed1cde None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:32.367088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1c8a623c-9716-4fa9-8f06-492842ed1cde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:32.514575 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8a623c-9716-4fa9-8f06-492842ed1cde None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:32.542567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:32.630103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666ffb6-bad4-4f08-bc27-5815b7a53098 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:32.630654 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666ffb6-bad4-4f08-bc27-5815b7a53098 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:32.630936 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666ffb6-bad4-4f08-bc27-5815b7a53098 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:32.631207 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666ffb6-bad4-4f08-bc27-5815b7a53098 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:32.632820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3666ffb6-bad4-4f08-bc27-5815b7a53098 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:32.722681 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666ffb6-bad4-4f08-bc27-5815b7a53098 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:32.724312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3666ffb6-bad4-4f08-bc27-5815b7a53098 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:32.742599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 722/1440] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:34.598445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed95d0-6df9-44f8-8a22-249c05dc2cf4 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:34.599042 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed95d0-6df9-44f8-8a22-249c05dc2cf4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:34.599042 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ed95d0-6df9-44f8-8a22-249c05dc2cf4 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:34.600346 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:20:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:34.605882 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928180a5-36fe-4178-9839-8c4383a9e75f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:34.606321 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928180a5-36fe-4178-9839-8c4383a9e75f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:34.606578 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928180a5-36fe-4178-9839-8c4383a9e75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:34.649180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:34.676179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-928180a5-36fe-4178-9839-8c4383a9e75f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:34.677407 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-928180a5-36fe-4178-9839-8c4383a9e75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:34.773618 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:20:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:35.065025 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed95d0-6df9-44f8-8a22-249c05dc2cf4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:35.066592 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e4ed95d0-6df9-44f8-8a22-249c05dc2cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:35.174350 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ed95d0-6df9-44f8-8a22-249c05dc2cf4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:35.206596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:35.249168 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77957dbb-3c79-40fd-be29-e1beb801f57e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:35.249812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77957dbb-3c79-40fd-be29-e1beb801f57e service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:35.250093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77957dbb-3c79-40fd-be29-e1beb801f57e service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:35.250418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77957dbb-3c79-40fd-be29-e1beb801f57e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:35.252067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-77957dbb-3c79-40fd-be29-e1beb801f57e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:35.363266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77957dbb-3c79-40fd-be29-e1beb801f57e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:35.364745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77957dbb-3c79-40fd-be29-e1beb801f57e service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:35.386933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 720/1443] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:37.673630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cc23a-1952-40f3-8452-4d008a8a35ed None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:37.674176 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cc23a-1952-40f3-8452-4d008a8a35ed None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:37.674176 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2cc23a-1952-40f3-8452-4d008a8a35ed None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:37.675599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:20:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:37.681155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fda52-384c-4ca2-87a9-3d9fff33bbb3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:37.681583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fda52-384c-4ca2-87a9-3d9fff33bbb3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:37.681698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4fda52-384c-4ca2-87a9-3d9fff33bbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:37.747277 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ef4fda52-384c-4ca2-87a9-3d9fff33bbb3 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:37.748512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ef4fda52-384c-4ca2-87a9-3d9fff33bbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:37.761780 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:37.846741 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:38.137929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cc23a-1952-40f3-8452-4d008a8a35ed None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:38.139538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-df2cc23a-1952-40f3-8452-4d008a8a35ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:38.243872 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2cc23a-1952-40f3-8452-4d008a8a35ed None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:38.256058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:38.317351 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7679a859-f7ba-4dab-9969-e06dda5c6dbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:38.317956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679a859-f7ba-4dab-9969-e06dda5c6dbd service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:38.318261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679a859-f7ba-4dab-9969-e06dda5c6dbd service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:38.318534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7679a859-f7ba-4dab-9969-e06dda5c6dbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:38.320126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7679a859-f7ba-4dab-9969-e06dda5c6dbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:38.427559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679a859-f7ba-4dab-9969-e06dda5c6dbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:38.429037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7679a859-f7ba-4dab-9969-e06dda5c6dbd service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:38.447516 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 725/1446] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:40.783009 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe86ac3-a392-42c6-85a7-307a7947eb28 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:40.783571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe86ac3-a392-42c6-85a7-307a7947eb28 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:40.783571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe86ac3-a392-42c6-85a7-307a7947eb28 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:40.784909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:20:40.790733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fa6b0-fb46-4280-9afd-ad4f039c759c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:40.791474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fa6b0-fb46-4280-9afd-ad4f039c759c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:40.791474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6fa6b0-fb46-4280-9afd-ad4f039c759c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:40.839772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:40.858245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4a6fa6b0-fb46-4280-9afd-ad4f039c759c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:40.859491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4a6fa6b0-fb46-4280-9afd-ad4f039c759c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:40.956209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:41.241283 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe86ac3-a392-42c6-85a7-307a7947eb28 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:41.242951 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6fe86ac3-a392-42c6-85a7-307a7947eb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:41.347194 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe86ac3-a392-42c6-85a7-307a7947eb28 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:41.387552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:41.422523 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e4c3aa-121c-49cf-bff4-0353e56e42cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:41.423137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4c3aa-121c-49cf-bff4-0353e56e42cd service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:41.423421 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4c3aa-121c-49cf-bff4-0353e56e42cd service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:41.423695 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e4c3aa-121c-49cf-bff4-0353e56e42cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:41.425296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-66e4c3aa-121c-49cf-bff4-0353e56e42cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:41.546809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e4c3aa-121c-49cf-bff4-0353e56e42cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:41.548229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e4c3aa-121c-49cf-bff4-0353e56e42cd service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:41.566155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 723/1449] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:43.630078 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc60255-b70c-48f6-948f-15e2b6fb7ae6 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:43.630666 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc60255-b70c-48f6-948f-15e2b6fb7ae6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:43.630666 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc60255-b70c-48f6-948f-15e2b6fb7ae6 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:43.632003 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:43.637882 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ea602-a9d6-45c7-ba47-b5607322fc82 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:43.638321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ea602-a9d6-45c7-ba47-b5607322fc82 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:43.638468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ea602-a9d6-45c7-ba47-b5607322fc82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:43.706731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-847ea602-a9d6-45c7-ba47-b5607322fc82 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:43.707956 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-847ea602-a9d6-45c7-ba47-b5607322fc82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:43.729806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:43.807627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:20:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:44.094228 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc60255-b70c-48f6-948f-15e2b6fb7ae6 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:44.095792 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5dc60255-b70c-48f6-948f-15e2b6fb7ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:44.200841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc60255-b70c-48f6-948f-15e2b6fb7ae6 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:44.221152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:44.276134 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa14cc4-807c-4eee-be95-1f9a8cad0371 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:44.276735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa14cc4-807c-4eee-be95-1f9a8cad0371 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:44.277012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa14cc4-807c-4eee-be95-1f9a8cad0371 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:44.277283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa14cc4-807c-4eee-be95-1f9a8cad0371 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:44.279172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8aa14cc4-807c-4eee-be95-1f9a8cad0371 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:44.386257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa14cc4-807c-4eee-be95-1f9a8cad0371 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:44.387639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa14cc4-807c-4eee-be95-1f9a8cad0371 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:44.406393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 728/1452] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:46.382266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4594c-49b3-40c1-9f36-90b17560821a None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:46.382776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4594c-49b3-40c1-9f36-90b17560821a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:46.382776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa4594c-49b3-40c1-9f36-90b17560821a None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:46.384083 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:46.389852 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27376bb-8ace-4c75-9a32-dfa515673242 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:46.390271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27376bb-8ace-4c75-9a32-dfa515673242 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:46.390443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27376bb-8ace-4c75-9a32-dfa515673242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:46.455496 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:46.458659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d27376bb-8ace-4c75-9a32-dfa515673242 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:46.460883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d27376bb-8ace-4c75-9a32-dfa515673242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:46.566542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:20:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:46.858040 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4594c-49b3-40c1-9f36-90b17560821a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:46.859967 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6aa4594c-49b3-40c1-9f36-90b17560821a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:46.976897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa4594c-49b3-40c1-9f36-90b17560821a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:47.042206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:47.066807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f357d-0455-4d00-b7b1-90b61c64cc29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:47.067413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f357d-0455-4d00-b7b1-90b61c64cc29 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:47.067747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f357d-0455-4d00-b7b1-90b61c64cc29 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:47.067962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f357d-0455-4d00-b7b1-90b61c64cc29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:47.069574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6c3f357d-0455-4d00-b7b1-90b61c64cc29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:47.182987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f357d-0455-4d00-b7b1-90b61c64cc29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:47.184619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f357d-0455-4d00-b7b1-90b61c64cc29 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:47.204818 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 726/1455] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:49.186381 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010e1fe-bae8-402f-8517-bca86f5050f1 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:49.186925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010e1fe-bae8-402f-8517-bca86f5050f1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:49.186925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010e1fe-bae8-402f-8517-bca86f5050f1 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:49.188283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:49.193916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94e37c-7175-43ac-a58d-dd5f2f3b2288 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:49.194351 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94e37c-7175-43ac-a58d-dd5f2f3b2288 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:49.194544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb94e37c-7175-43ac-a58d-dd5f2f3b2288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:49.261710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-eb94e37c-7175-43ac-a58d-dd5f2f3b2288 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:49.262954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-eb94e37c-7175-43ac-a58d-dd5f2f3b2288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:49.292809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:49.361828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:49.643783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010e1fe-bae8-402f-8517-bca86f5050f1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:49.645317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6010e1fe-bae8-402f-8517-bca86f5050f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:49.752656 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010e1fe-bae8-402f-8517-bca86f5050f1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:49.784147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:49.830845 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8da395-c4ba-4428-8ec7-cf134e86c829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:49.831454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8da395-c4ba-4428-8ec7-cf134e86c829 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:49.831735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8da395-c4ba-4428-8ec7-cf134e86c829 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:49.832005 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8da395-c4ba-4428-8ec7-cf134e86c829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:49.833616 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4a8da395-c4ba-4428-8ec7-cf134e86c829 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:49.960359 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8da395-c4ba-4428-8ec7-cf134e86c829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:49.961856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8da395-c4ba-4428-8ec7-cf134e86c829 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:49.979949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 731/1458] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:52.066013 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c17e71-6f91-4f82-ba9c-6641b7bc3fa0 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:52.066658 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c17e71-6f91-4f82-ba9c-6641b7bc3fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:52.066767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c17e71-6f91-4f82-ba9c-6641b7bc3fa0 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:52.068159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:52.073984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4adcfb-b839-4cc2-b067-8296e254f1dc None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:52.074497 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4adcfb-b839-4cc2-b067-8296e254f1dc None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:52.074600 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4adcfb-b839-4cc2-b067-8296e254f1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:52.147877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3b4adcfb-b839-4cc2-b067-8296e254f1dc None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:52.149604 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3b4adcfb-b839-4cc2-b067-8296e254f1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:52.158376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:52.268621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:52.549212 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c17e71-6f91-4f82-ba9c-6641b7bc3fa0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:52.550792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c9c17e71-6f91-4f82-ba9c-6641b7bc3fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:52.772225 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c17e71-6f91-4f82-ba9c-6641b7bc3fa0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:52.831976 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:52.850112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88482be-08d5-46ab-891d-f78fb54fb0f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:52.850742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88482be-08d5-46ab-891d-f78fb54fb0f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:52.851017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88482be-08d5-46ab-891d-f78fb54fb0f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:52.851287 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88482be-08d5-46ab-891d-f78fb54fb0f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:52.853407 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c88482be-08d5-46ab-891d-f78fb54fb0f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:52.970353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88482be-08d5-46ab-891d-f78fb54fb0f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:52.971827 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88482be-08d5-46ab-891d-f78fb54fb0f3 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:52.990818 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 729/1461] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:54.975107 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421abba-79b4-4108-b0c1-dc0e29165c9b None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:54.975686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421abba-79b4-4108-b0c1-dc0e29165c9b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:54.975686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421abba-79b4-4108-b0c1-dc0e29165c9b None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:54.977046 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:20:54.982596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d824b-47a2-4567-b7d9-636ba8435bd6 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:54.983414 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d824b-47a2-4567-b7d9-636ba8435bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:54.983414 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d824b-47a2-4567-b7d9-636ba8435bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:55.051011 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cd2d824b-47a2-4567-b7d9-636ba8435bd6 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:55.052233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cd2d824b-47a2-4567-b7d9-636ba8435bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:55.092598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:55.152674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:20:55.441557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421abba-79b4-4108-b0c1-dc0e29165c9b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:55.443225 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f421abba-79b4-4108-b0c1-dc0e29165c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:55.552912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421abba-79b4-4108-b0c1-dc0e29165c9b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:55.594236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:55.630450 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543d138-bb76-47d0-9a76-6fe575956e8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:55.631072 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543d138-bb76-47d0-9a76-6fe575956e8c service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:55.631344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543d138-bb76-47d0-9a76-6fe575956e8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:55.631605 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543d138-bb76-47d0-9a76-6fe575956e8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:55.633222 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e543d138-bb76-47d0-9a76-6fe575956e8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:55.746553 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543d138-bb76-47d0-9a76-6fe575956e8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:55.747898 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543d138-bb76-47d0-9a76-6fe575956e8c service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:55.765591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 734/1464] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:20:57.775891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ab97e-2490-4d8a-ad80-e11838b3e474 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:57.776506 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ab97e-2490-4d8a-ad80-e11838b3e474 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:57.776506 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2ab97e-2490-4d8a-ad80-e11838b3e474 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:57.777820 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:20:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:20:57.783774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef194f7b-960b-4c0a-a27e-c5f6d8719eea None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:57.784232 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef194f7b-960b-4c0a-a27e-c5f6d8719eea None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:57.784391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef194f7b-960b-4c0a-a27e-c5f6d8719eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:57.854051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ef194f7b-960b-4c0a-a27e-c5f6d8719eea None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:20:57.855294 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ef194f7b-960b-4c0a-a27e-c5f6d8719eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:57.873658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:57.965080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:20:58.256460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ab97e-2490-4d8a-ad80-e11838b3e474 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:20:58.258015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cd2ab97e-2490-4d8a-ad80-e11838b3e474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:58.378930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2ab97e-2490-4d8a-ad80-e11838b3e474 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:20:58.446168 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:20:58.453701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621eb49a-778e-40ca-a5ac-bd7d7cdb43ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:20:58.454371 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eb49a-778e-40ca-a5ac-bd7d7cdb43ea service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:20:58.454708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eb49a-778e-40ca-a5ac-bd7d7cdb43ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:20:58.454974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621eb49a-778e-40ca-a5ac-bd7d7cdb43ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:20:58.456565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-621eb49a-778e-40ca-a5ac-bd7d7cdb43ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:20:58.566358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eb49a-778e-40ca-a5ac-bd7d7cdb43ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:20:58.567802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621eb49a-778e-40ca-a5ac-bd7d7cdb43ea service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:20:58.586913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 732/1467] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:20:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:21:00.527683 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60105d-1243-4a23-a75e-4af3c2cec4aa None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:00.528535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60105d-1243-4a23-a75e-4af3c2cec4aa None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:00.528535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60105d-1243-4a23-a75e-4af3c2cec4aa None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:00.529817 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:21:00.535555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b938214-7cc9-4f55-9153-d29a1ed1aa16 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:00.536003 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b938214-7cc9-4f55-9153-d29a1ed1aa16 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:00.536097 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b938214-7cc9-4f55-9153-d29a1ed1aa16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:00.605676 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0b938214-7cc9-4f55-9153-d29a1ed1aa16 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:21:00.606954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0b938214-7cc9-4f55-9153-d29a1ed1aa16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:00.656383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:00.708486 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:21:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:21:00.992291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60105d-1243-4a23-a75e-4af3c2cec4aa None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:21:00.993985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bd60105d-1243-4a23-a75e-4af3c2cec4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:01.098185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd60105d-1243-4a23-a75e-4af3c2cec4aa None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:21:01.146784 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:01.173809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfdb88-876a-494c-80f9-de2fcbbd1eb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:21:01.174443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfdb88-876a-494c-80f9-de2fcbbd1eb4 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:01.174724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfdb88-876a-494c-80f9-de2fcbbd1eb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:01.174996 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfdb88-876a-494c-80f9-de2fcbbd1eb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:01.176611 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b1cfdb88-876a-494c-80f9-de2fcbbd1eb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:01.289233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfdb88-876a-494c-80f9-de2fcbbd1eb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:21:01.290635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfdb88-876a-494c-80f9-de2fcbbd1eb4 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:21:01.309310 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 737/1470] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:21:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:21:10.706513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=22 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:11.307129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=66 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:20.987309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb00f9e-c1ae-4dd6-92e1-60409dd2f281 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:20.987989 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb00f9e-c1ae-4dd6-92e1-60409dd2f281 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:20.987989 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb00f9e-c1ae-4dd6-92e1-60409dd2f281 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:20.989193 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 734/1471] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:21:20.994897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb593961-39bb-4044-8dad-8efe3983b79f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:20.998189 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb593961-39bb-4044-8dad-8efe3983b79f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:20.998189 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb593961-39bb-4044-8dad-8efe3983b79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:21.071049 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cb593961-39bb-4044-8dad-8efe3983b79f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:21:21.072328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cb593961-39bb-4044-8dad-8efe3983b79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:21.178464 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 738/1472] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:21:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:21:21.483623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2eb0ff-b418-434e-900f-9b7b6c2ced7b None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:21.483909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2eb0ff-b418-434e-900f-9b7b6c2ced7b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:21.484180 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2eb0ff-b418-434e-900f-9b7b6c2ced7b None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:21.485520 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 735/1473] 10.222.0.76 () {54 vars in 967 bytes} [Wed Aug 24 15:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:21:21.491305 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be02bdd-7a67-45d2-b9ed-01a9036f316d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:21.491617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be02bdd-7a67-45d2-b9ed-01a9036f316d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:21.492458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be02bdd-7a67-45d2-b9ed-01a9036f316d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:21.576271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9be02bdd-7a67-45d2-b9ed-01a9036f316d None None] MFA Rules not processed for user `96c71ea88d9a474c9ff7a30555cfa91f`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:21:21.577545 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9be02bdd-7a67-45d2-b9ed-01a9036f316d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:21.598926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:21.666899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 739/1474] 10.222.0.76 () {58 vars in 1064 bytes} [Wed Aug 24 15:21:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:21:21.675001 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2eb0ff-b418-434e-900f-9b7b6c2ced7b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:21:21.676549 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6a2eb0ff-b418-434e-900f-9b7b6c2ced7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:21.786040 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2eb0ff-b418-434e-900f-9b7b6c2ced7b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:21:21.859901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1862bb75-0998-4276-b3bf-a16d5d14c1b1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96c71ea88d9a474c9ff7a30555cfa91f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:21:21.860581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862bb75-0998-4276-b3bf-a16d5d14c1b1 service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:21.860773 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862bb75-0998-4276-b3bf-a16d5d14c1b1 service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:21.861036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1862bb75-0998-4276-b3bf-a16d5d14c1b1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:21.862663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1862bb75-0998-4276-b3bf-a16d5d14c1b1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:21.917251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:21.972023 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862bb75-0998-4276-b3bf-a16d5d14c1b1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:21:21.973445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1862bb75-0998-4276-b3bf-a16d5d14c1b1 service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:21:21.993238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 736/1475] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:21:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:21:24.208026 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcd6a2-f2bc-4e59-85f8-0f3a25f45b34 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:24.208622 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcd6a2-f2bc-4e59-85f8-0f3a25f45b34 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:24.208800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efcd6a2-f2bc-4e59-85f8-0f3a25f45b34 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:24.210426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 740/1476] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:21:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:21:24.215619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb7d9b-d244-416e-9523-715719ddc502 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:24.216089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb7d9b-d244-416e-9523-715719ddc502 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:24.216196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb7d9b-d244-416e-9523-715719ddc502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:24.306916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4fcb7d9b-d244-416e-9523-715719ddc502 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:21:24.308132 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4fcb7d9b-d244-416e-9523-715719ddc502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:24.404341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 737/1477] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:21:24.683575 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efcd6a2-f2bc-4e59-85f8-0f3a25f45b34 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:21:24.685114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1efcd6a2-f2bc-4e59-85f8-0f3a25f45b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:24.829186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efcd6a2-f2bc-4e59-85f8-0f3a25f45b34 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:21:24.888988 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:24.941571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81b9cec-d729-4188-94a7-d5b4927e2296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:21:24.942178 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b9cec-d729-4188-94a7-d5b4927e2296 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:24.942584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b9cec-d729-4188-94a7-d5b4927e2296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:24.942786 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81b9cec-d729-4188-94a7-d5b4927e2296 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:24.943396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b9cec-d729-4188-94a7-d5b4927e2296 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:21:24.945723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: WARNING py.warnings [None req-e81b9cec-d729-4188-94a7-d5b4927e2296 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:21:24.945723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: warnings.warn(msg) Aug 24 15:21:24.945723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: Aug 24 15:21:24.946908 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81b9cec-d729-4188-94a7-d5b4927e2296 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:21:24.967302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 741/1478] 10.222.0.76 () {56 vars in 1108 bytes} [Wed Aug 24 15:21:24 2022] GET /identity/v3/projects => generated 2553 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:21:26.894667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568c25d-ca28-4015-bdea-d47b2b1bbf65 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:26.895196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568c25d-ca28-4015-bdea-d47b2b1bbf65 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:26.895196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568c25d-ca28-4015-bdea-d47b2b1bbf65 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:26.896453 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 738/1479] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:21:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:21:26.901850 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321a911-5889-4116-86c7-80831d6b3042 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:26.902332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321a911-5889-4116-86c7-80831d6b3042 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:26.902456 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321a911-5889-4116-86c7-80831d6b3042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:26.970649 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6321a911-5889-4116-86c7-80831d6b3042 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:21:26.971878 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6321a911-5889-4116-86c7-80831d6b3042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:27.081469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 742/1480] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:21:27.375928 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568c25d-ca28-4015-bdea-d47b2b1bbf65 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:21:27.377546 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8568c25d-ca28-4015-bdea-d47b2b1bbf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:27.387385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:27.528182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568c25d-ca28-4015-bdea-d47b2b1bbf65 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:21:27.613044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:27.647824 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b48886-b686-4820-b041-7bfec52c664b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:21:27.648964 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b48886-b686-4820-b041-7bfec52c664b service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:27.649243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b48886-b686-4820-b041-7bfec52c664b service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:27.649535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b48886-b686-4820-b041-7bfec52c664b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:27.651182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-41b48886-b686-4820-b041-7bfec52c664b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:27.759240 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b48886-b686-4820-b041-7bfec52c664b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:21:27.760823 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b48886-b686-4820-b041-7bfec52c664b service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:21:27.780305 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 739/1481] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:21:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:21:29.862193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98664f71-b1c5-41a1-bbbc-e1641d26589d None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:29.862772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98664f71-b1c5-41a1-bbbc-e1641d26589d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:29.862772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98664f71-b1c5-41a1-bbbc-e1641d26589d None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:29.864053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 743/1482] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:21:29.869461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a832b-fee7-47a7-bfc5-107053cbe540 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:29.869901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a832b-fee7-47a7-bfc5-107053cbe540 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:29.870008 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9a832b-fee7-47a7-bfc5-107053cbe540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:29.938732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4e9a832b-fee7-47a7-bfc5-107053cbe540 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:21:29.939968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4e9a832b-fee7-47a7-bfc5-107053cbe540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:29.989376 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:30.039813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 740/1483] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:21:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:21:30.325957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98664f71-b1c5-41a1-bbbc-e1641d26589d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:21:30.327566 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-98664f71-b1c5-41a1-bbbc-e1641d26589d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:30.351475 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:30.433559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98664f71-b1c5-41a1-bbbc-e1641d26589d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:21:30.523677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f48f9-9c4f-4ed5-bdb0-f87ddf23e3b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:21:30.524285 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f48f9-9c4f-4ed5-bdb0-f87ddf23e3b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:30.524563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f48f9-9c4f-4ed5-bdb0-f87ddf23e3b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:30.524836 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f48f9-9c4f-4ed5-bdb0-f87ddf23e3b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:30.526482 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5f2f48f9-9c4f-4ed5-bdb0-f87ddf23e3b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:30.601177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:30.637486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f48f9-9c4f-4ed5-bdb0-f87ddf23e3b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:21:30.638958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2f48f9-9c4f-4ed5-bdb0-f87ddf23e3b0 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:21:30.657388 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 744/1484] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:21:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:21:32.653642 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7552a-563f-4dcc-8ccf-e195acedec43 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:32.654579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7552a-563f-4dcc-8ccf-e195acedec43 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:32.654579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7552a-563f-4dcc-8ccf-e195acedec43 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:32.655797 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 741/1485] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:21:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:21:32.662164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92424ddc-f71e-4ac3-b791-85f52db055eb None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:32.662715 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92424ddc-f71e-4ac3-b791-85f52db055eb None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:32.663044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92424ddc-f71e-4ac3-b791-85f52db055eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:32.742945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-92424ddc-f71e-4ac3-b791-85f52db055eb None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:21:32.744244 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-92424ddc-f71e-4ac3-b791-85f52db055eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:32.846820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 745/1486] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:21:33.196752 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7552a-563f-4dcc-8ccf-e195acedec43 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:21:33.198336 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4ec7552a-563f-4dcc-8ccf-e195acedec43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:33.306199 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7552a-563f-4dcc-8ccf-e195acedec43 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:21:33.384135 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:33.386929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5d4845-8dd7-4c3f-88a8-102746bb9a8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:21:33.387734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d4845-8dd7-4c3f-88a8-102746bb9a8f service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:33.388023 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d4845-8dd7-4c3f-88a8-102746bb9a8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:33.389005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5d4845-8dd7-4c3f-88a8-102746bb9a8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:33.391276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9d5d4845-8dd7-4c3f-88a8-102746bb9a8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:33.510584 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d4845-8dd7-4c3f-88a8-102746bb9a8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:21:33.512700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5d4845-8dd7-4c3f-88a8-102746bb9a8f service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:21:33.535807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 742/1487] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:21:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:21:35.680320 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743aa429-3c27-4280-a632-eabeac2aa29b None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:35.680860 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743aa429-3c27-4280-a632-eabeac2aa29b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:35.680996 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743aa429-3c27-4280-a632-eabeac2aa29b None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:35.682624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 746/1488] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:21:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:21:35.689062 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c231e-f1d8-4206-9df4-0b644dbc540a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:35.689500 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c231e-f1d8-4206-9df4-0b644dbc540a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:35.689603 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0c231e-f1d8-4206-9df4-0b644dbc540a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:35.770829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2d0c231e-f1d8-4206-9df4-0b644dbc540a None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:21:35.771964 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2d0c231e-f1d8-4206-9df4-0b644dbc540a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:35.829748 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:35.875005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 743/1489] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:21:36.210174 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743aa429-3c27-4280-a632-eabeac2aa29b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:21:36.211984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-743aa429-3c27-4280-a632-eabeac2aa29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:36.248762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:36.325960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743aa429-3c27-4280-a632-eabeac2aa29b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:21:36.414798 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952dd04b-84a7-4c4b-b112-973dedebbc7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:21:36.415413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952dd04b-84a7-4c4b-b112-973dedebbc7a service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:36.415693 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952dd04b-84a7-4c4b-b112-973dedebbc7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:36.415960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952dd04b-84a7-4c4b-b112-973dedebbc7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:36.417602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-952dd04b-84a7-4c4b-b112-973dedebbc7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:36.504551 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:36.542585 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952dd04b-84a7-4c4b-b112-973dedebbc7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:21:36.544047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952dd04b-84a7-4c4b-b112-973dedebbc7a service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:21:36.562978 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 747/1490] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:21:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:21:38.772544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427a610-2686-49f7-ba6b-ee9aff594cc6 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:38.773639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427a610-2686-49f7-ba6b-ee9aff594cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:38.773639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0427a610-2686-49f7-ba6b-ee9aff594cc6 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:38.775064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 744/1491] 10.222.0.76 () {54 vars in 927 bytes} [Wed Aug 24 15:21:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:21:38.781055 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b59c26-489b-4020-b2ca-c9d8623ae030 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:38.781483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b59c26-489b-4020-b2ca-c9d8623ae030 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:38.781678 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b59c26-489b-4020-b2ca-c9d8623ae030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:38.849789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e3b59c26-489b-4020-b2ca-c9d8623ae030 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:21:38.851451 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e3b59c26-489b-4020-b2ca-c9d8623ae030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:38.964297 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 748/1492] 10.222.0.76 () {58 vars in 1024 bytes} [Wed Aug 24 15:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:21:39.256227 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427a610-2686-49f7-ba6b-ee9aff594cc6 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:21:39.258602 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0427a610-2686-49f7-ba6b-ee9aff594cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:39.379852 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0427a610-2686-49f7-ba6b-ee9aff594cc6 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:21:39.461517 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fc869-31fc-45d9-9618-b1015b88aac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:21:39.462159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fc869-31fc-45d9-9618-b1015b88aac4 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:21:39.462470 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fc869-31fc-45d9-9618-b1015b88aac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:21:39.462738 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fc869-31fc-45d9-9618-b1015b88aac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:21:39.464389 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7e8fc869-31fc-45d9-9618-b1015b88aac4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:21:39.475113 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:39.582994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fc869-31fc-45d9-9618-b1015b88aac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:21:39.585043 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fc869-31fc-45d9-9618-b1015b88aac4 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:21:39.606763 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 745/1493] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:21:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:21:48.962333 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=82 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:21:49.604108 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=50 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:22:51.402633 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0cb876-b65d-4669-9e33-1bcca6ee05d7 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:51.403257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0cb876-b65d-4669-9e33-1bcca6ee05d7 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:51.403671 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0cb876-b65d-4669-9e33-1bcca6ee05d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:51.504930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0e0cb876-b65d-4669-9e33-1bcca6ee05d7 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:22:51.506173 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0e0cb876-b65d-4669-9e33-1bcca6ee05d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:22:51.608297 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 749/1494] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:22:51.616647 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01e31e-d808-4cf3-9ffc-a7a7ca3cd968 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:51.617166 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01e31e-d808-4cf3-9ffc-a7a7ca3cd968 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:51.617166 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db01e31e-d808-4cf3-9ffc-a7a7ca3cd968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:51.707416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-db01e31e-d808-4cf3-9ffc-a7a7ca3cd968 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:22:51.709194 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-db01e31e-d808-4cf3-9ffc-a7a7ca3cd968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:22:51.812204 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 746/1495] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:22:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:22:51.817557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0cb876-b65d-4669-9e33-1bcca6ee05d7 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:22:51.819427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0e0cb876-b65d-4669-9e33-1bcca6ee05d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:22:51.831475 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0cb876-b65d-4669-9e33-1bcca6ee05d7 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:22:51.897570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:22:51.934457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77161412-958a-40d7-985e-1b6f27356ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:22:51.935155 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77161412-958a-40d7-985e-1b6f27356ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:51.935428 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77161412-958a-40d7-985e-1b6f27356ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:51.935692 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77161412-958a-40d7-985e-1b6f27356ad4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:51.936368 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77161412-958a-40d7-985e-1b6f27356ad4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:22:51.936785 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77161412-958a-40d7-985e-1b6f27356ad4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:22:51.937652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: WARNING py.warnings [None req-77161412-958a-40d7-985e-1b6f27356ad4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:22:51.937652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: warnings.warn(msg) Aug 24 15:22:51.937652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: Aug 24 15:22:51.938636 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77161412-958a-40d7-985e-1b6f27356ad4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:22:51.954816 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 750/1496] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:22:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:22:51.964212 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01e31e-d808-4cf3-9ffc-a7a7ca3cd968 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:22:51.973911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db01e31e-d808-4cf3-9ffc-a7a7ca3cd968 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:22:52.041953 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:22:52.079004 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77029b7a-d285-4d46-aa6b-88771a004925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:22:52.079680 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77029b7a-d285-4d46-aa6b-88771a004925 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:52.079957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77029b7a-d285-4d46-aa6b-88771a004925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:52.080190 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77029b7a-d285-4d46-aa6b-88771a004925 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:52.080844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77029b7a-d285-4d46-aa6b-88771a004925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:22:52.082681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-77029b7a-d285-4d46-aa6b-88771a004925 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:22:52.082681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 24 15:22:52.082681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: Aug 24 15:22:52.084431 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77029b7a-d285-4d46-aa6b-88771a004925 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:22:52.147150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 747/1497] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:22:51 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:22:52.152412 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77161412-958a-40d7-985e-1b6f27356ad4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:22:52.154341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-77161412-958a-40d7-985e-1b6f27356ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:22:52.191926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77161412-958a-40d7-985e-1b6f27356ad4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:22:52.300838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af8513-1510-4f94-8904-bc9788086f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:22:52.301352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af8513-1510-4f94-8904-bc9788086f1b admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:52.301625 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af8513-1510-4f94-8904-bc9788086f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:52.301888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4af8513-1510-4f94-8904-bc9788086f1b admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:52.303035 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af8513-1510-4f94-8904-bc9788086f1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:22:52.304057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: WARNING py.warnings [None req-f4af8513-1510-4f94-8904-bc9788086f1b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:22:52.304057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: warnings.warn(msg) Aug 24 15:22:52.304057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: Aug 24 15:22:52.305269 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4af8513-1510-4f94-8904-bc9788086f1b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:22:52.351520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 751/1498] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:22:52 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:22:52.356027 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77029b7a-d285-4d46-aa6b-88771a004925 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:22:52.357601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-77029b7a-d285-4d46-aa6b-88771a004925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:22:52.422660 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77029b7a-d285-4d46-aa6b-88771a004925 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:22:52.510615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:22:52.523410 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d9ede-13ba-493f-a05e-7dc6d4b580f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:22:52.524042 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d9ede-13ba-493f-a05e-7dc6d4b580f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:52.524315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d9ede-13ba-493f-a05e-7dc6d4b580f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:52.524591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d9ede-13ba-493f-a05e-7dc6d4b580f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:52.525329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d9ede-13ba-493f-a05e-7dc6d4b580f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:22:52.526458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-923d9ede-13ba-493f-a05e-7dc6d4b580f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:22:52.526458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 24 15:22:52.526458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: Aug 24 15:22:52.527348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923d9ede-13ba-493f-a05e-7dc6d4b580f6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:22:52.542526 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 748/1499] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:22:52.546536 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af8513-1510-4f94-8904-bc9788086f1b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:22:52.548055 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f4af8513-1510-4f94-8904-bc9788086f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:22:52.561381 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4af8513-1510-4f94-8904-bc9788086f1b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:22:52.587157 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:22:52.661379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f1c84d-edc1-4d6d-ab42-794099fb8a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:22:52.663599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1c84d-edc1-4d6d-ab42-794099fb8a77 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:52.663599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1c84d-edc1-4d6d-ab42-794099fb8a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:52.663599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f1c84d-edc1-4d6d-ab42-794099fb8a77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:52.664530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1c84d-edc1-4d6d-ab42-794099fb8a77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:22:52.664735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: WARNING py.warnings [None req-92f1c84d-edc1-4d6d-ab42-794099fb8a77 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:22:52.664735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: warnings.warn(msg) Aug 24 15:22:52.664735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: Aug 24 15:22:52.665640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f1c84d-edc1-4d6d-ab42-794099fb8a77 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:22:52.679727 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 752/1500] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:22:52 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:22:52.683993 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d9ede-13ba-493f-a05e-7dc6d4b580f6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:22:52.685562 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-923d9ede-13ba-493f-a05e-7dc6d4b580f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:22:52.719353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923d9ede-13ba-493f-a05e-7dc6d4b580f6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:22:52.821612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1adf257d-0970-4b50-9c98-e07c82889251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:22:52.822400 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf257d-0970-4b50-9c98-e07c82889251 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:52.822840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf257d-0970-4b50-9c98-e07c82889251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:52.823140 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1adf257d-0970-4b50-9c98-e07c82889251 admin admin] PATH_INFO: `/v3/projects/8b1282f6f0a04da287003ed37373a0dc/users/925dc100a06f4ccda2e59eb56fd70e05/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:52.851820 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf257d-0970-4b50-9c98-e07c82889251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b1282f6f0a04da287003ed37373a0dc, user_id=925dc100a06f4ccda2e59eb56fd70e05, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:22:52.852977 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-1adf257d-0970-4b50-9c98-e07c82889251 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:22:52.852977 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 24 15:22:52.852977 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: Aug 24 15:22:52.855454 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1adf257d-0970-4b50-9c98-e07c82889251 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:22:52.883742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 749/1501] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:22:52 2022] PUT /identity/v3/projects/8b1282f6f0a04da287003ed37373a0dc/users/925dc100a06f4ccda2e59eb56fd70e05/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:22:52.896778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1e79c-1d96-4278-bac3-7a42d1680a7f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:52.897009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1e79c-1d96-4278-bac3-7a42d1680a7f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:52.897276 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac1e79c-1d96-4278-bac3-7a42d1680a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:52.959882 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-eac1e79c-1d96-4278-bac3-7a42d1680a7f None None] MFA Rules not processed for user `925dc100a06f4ccda2e59eb56fd70e05`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:22:52.961104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-eac1e79c-1d96-4278-bac3-7a42d1680a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:22:53.044447 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 753/1502] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:22:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:22:53.057851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b162cb6-be01-4aec-8614-eedc3f20a390 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:53.058149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b162cb6-be01-4aec-8614-eedc3f20a390 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:53.058455 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b162cb6-be01-4aec-8614-eedc3f20a390 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:53.059777 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 750/1503] 10.222.0.76 () {54 vars in 964 bytes} [Wed Aug 24 15:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:22:53.064387 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098fd7d-83f2-4521-b975-90db28aa1863 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:53.064676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098fd7d-83f2-4521-b975-90db28aa1863 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:53.064953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098fd7d-83f2-4521-b975-90db28aa1863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:53.097817 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:22:53.141882 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8098fd7d-83f2-4521-b975-90db28aa1863 None None] MFA Rules not processed for user `194c7492cf954a6a8dd21de66aa24fec`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:22:53.143111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8098fd7d-83f2-4521-b975-90db28aa1863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:22:53.225920 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 754/1504] 10.222.0.76 () {58 vars in 1061 bytes} [Wed Aug 24 15:22:53 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:22:53.233827 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b162cb6-be01-4aec-8614-eedc3f20a390 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:22:53.235380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3b162cb6-be01-4aec-8614-eedc3f20a390 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:22:53.292611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:22:53.352080 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b162cb6-be01-4aec-8614-eedc3f20a390 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:22:53.428454 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65852d91-ad74-4b74-ba62-95051aa044bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:22:53.429064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65852d91-ad74-4b74-ba62-95051aa044bf service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:53.429340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65852d91-ad74-4b74-ba62-95051aa044bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:53.429606 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65852d91-ad74-4b74-ba62-95051aa044bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:53.431320 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-65852d91-ad74-4b74-ba62-95051aa044bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:22:53.542926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65852d91-ad74-4b74-ba62-95051aa044bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:22:53.544407 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65852d91-ad74-4b74-ba62-95051aa044bf service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:22:53.547974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:22:53.563341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 751/1505] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:22:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:22:53.609139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afc33e-27d7-4d60-a14b-9bb3a58e0a4b None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:53.609576 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afc33e-27d7-4d60-a14b-9bb3a58e0a4b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:53.609683 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94afc33e-27d7-4d60-a14b-9bb3a58e0a4b None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:53.610979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 755/1506] 10.222.0.76 () {52 vars in 830 bytes} [Wed Aug 24 15:22:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:22:53.615206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65852d91-ad74-4b74-ba62-95051aa044bf service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:22:53.616420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-65852d91-ad74-4b74-ba62-95051aa044bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:22:53.715412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65852d91-ad74-4b74-ba62-95051aa044bf service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:22:53.813781 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6383809-fde7-46d9-8cab-521aa366e0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:22:53.814432 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6383809-fde7-46d9-8cab-521aa366e0f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:53.814712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6383809-fde7-46d9-8cab-521aa366e0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:53.814990 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6383809-fde7-46d9-8cab-521aa366e0f7 admin admin] PATH_INFO: `/v3/users/925dc100a06f4ccda2e59eb56fd70e05` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:53.827843 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6383809-fde7-46d9-8cab-521aa366e0f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=925dc100a06f4ccda2e59eb56fd70e05)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:22:53.828828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-f6383809-fde7-46d9-8cab-521aa366e0f7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:22:53.828828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 24 15:22:53.828828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: Aug 24 15:22:53.830041 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6383809-fde7-46d9-8cab-521aa366e0f7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:22:53.839723 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:22:53.897885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f6383809-fde7-46d9-8cab-521aa366e0f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '925dc100a06f4ccda2e59eb56fd70e05'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:22:53.898373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f6383809-fde7-46d9-8cab-521aa366e0f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '925dc100a06f4ccda2e59eb56fd70e05'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:22:53.921346 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f6383809-fde7-46d9-8cab-521aa366e0f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '925dc100a06f4ccda2e59eb56fd70e05'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:22:53.931282 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f6383809-fde7-46d9-8cab-521aa366e0f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '925dc100a06f4ccda2e59eb56fd70e05'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:22:53.931720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f6383809-fde7-46d9-8cab-521aa366e0f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '925dc100a06f4ccda2e59eb56fd70e05'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:22:53.951455 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 752/1507] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:22:53 2022] DELETE /identity/v3/users/925dc100a06f4ccda2e59eb56fd70e05 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:22:53.956024 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2e958-0d47-410a-a951-cde462e0d489 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:53.956313 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2e958-0d47-410a-a951-cde462e0d489 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:53.956587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b2e958-0d47-410a-a951-cde462e0d489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:54.014553 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-95b2e958-0d47-410a-a951-cde462e0d489 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:22:54.015775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-95b2e958-0d47-410a-a951-cde462e0d489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:22:54.092767 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:22:54.224817 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 756/1508] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:22:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:22:54.234647 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6383809-fde7-46d9-8cab-521aa366e0f7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:22:54.235961 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f6383809-fde7-46d9-8cab-521aa366e0f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:22:54.336240 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6383809-fde7-46d9-8cab-521aa366e0f7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:22:54.394052 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:22:54.415258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1731d4b-2c55-4282-a59c-fa19f59932ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:22:54.415872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1731d4b-2c55-4282-a59c-fa19f59932ee service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:54.416148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1731d4b-2c55-4282-a59c-fa19f59932ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:54.416418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1731d4b-2c55-4282-a59c-fa19f59932ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:54.418063 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a1731d4b-2c55-4282-a59c-fa19f59932ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:22:54.532653 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1731d4b-2c55-4282-a59c-fa19f59932ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:22:54.533529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1731d4b-2c55-4282-a59c-fa19f59932ee service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:22:54.552824 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 753/1509] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:22:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:22:54.916546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2e958-0d47-410a-a951-cde462e0d489 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:22:54.918134 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-95b2e958-0d47-410a-a951-cde462e0d489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:22:54.929744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b2e958-0d47-410a-a951-cde462e0d489 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:22:55.028698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea9943-1fe3-4d2e-b367-03456b209b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:22:55.029328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea9943-1fe3-4d2e-b367-03456b209b08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:22:55.029599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea9943-1fe3-4d2e-b367-03456b209b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:22:55.030122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea9943-1fe3-4d2e-b367-03456b209b08 admin admin] PATH_INFO: `/v3/projects/8b1282f6f0a04da287003ed37373a0dc` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:22:55.040060 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea9943-1fe3-4d2e-b367-03456b209b08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b1282f6f0a04da287003ed37373a0dc)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:22:55.041083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: WARNING py.warnings [None req-40ea9943-1fe3-4d2e-b367-03456b209b08 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:22:55.041083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: warnings.warn(msg) Aug 24 15:22:55.041083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: Aug 24 15:22:55.042301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ea9943-1fe3-4d2e-b367-03456b209b08 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:22:55.100021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-40ea9943-1fe3-4d2e-b367-03456b209b08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b1282f6f0a04da287003ed37373a0dc', 'request_id': 'req-40ea9943-1fe3-4d2e-b367-03456b209b08'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:22:55.112947 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-40ea9943-1fe3-4d2e-b367-03456b209b08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b1282f6f0a04da287003ed37373a0dc', 'request_id': 'req-40ea9943-1fe3-4d2e-b367-03456b209b08'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:22:55.123110 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-40ea9943-1fe3-4d2e-b367-03456b209b08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b1282f6f0a04da287003ed37373a0dc', 'request_id': 'req-40ea9943-1fe3-4d2e-b367-03456b209b08'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:22:55.145820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-40ea9943-1fe3-4d2e-b367-03456b209b08 admin admin] The token cache is being invalidate because project 8b1282f6f0a04da287003ed37373a0dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:22:55.146105 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-40ea9943-1fe3-4d2e-b367-03456b209b08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:22:55.146568 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-40ea9943-1fe3-4d2e-b367-03456b209b08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:22:55.147962 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 757/1510] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:22:54 2022] DELETE /identity/v3/projects/8b1282f6f0a04da287003ed37373a0dc => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:23:04.550365 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=62 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:23:05.144269 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:27.602000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92640f-2c51-4c05-b477-bd541919a15c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:27.602628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92640f-2c51-4c05-b477-bd541919a15c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:27.602628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d92640f-2c51-4c05-b477-bd541919a15c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:27.603910 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:27.609314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113afe5-4899-46dd-828c-3e2e80503ddd None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:27.609820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113afe5-4899-46dd-828c-3e2e80503ddd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:27.609923 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b113afe5-4899-46dd-828c-3e2e80503ddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:27.689512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b113afe5-4899-46dd-828c-3e2e80503ddd None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:27.690762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b113afe5-4899-46dd-828c-3e2e80503ddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:27.793576 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:24:27.951095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92640f-2c51-4c05-b477-bd541919a15c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:27.953198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1d92640f-2c51-4c05-b477-bd541919a15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:28.073280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d92640f-2c51-4c05-b477-bd541919a15c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:28.078920 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3df3a-78b9-4267-b919-2208db0ae9bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:28.079528 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3df3a-78b9-4267-b919-2208db0ae9bd service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:28.079832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3df3a-78b9-4267-b919-2208db0ae9bd service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:28.080103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3df3a-78b9-4267-b919-2208db0ae9bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:28.088896 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3df3a-78b9-4267-b919-2208db0ae9bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:28.090492 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da3df3a-78b9-4267-b919-2208db0ae9bd service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:28.099829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 755/1513] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:24:28.936883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804ad1a-9225-4e37-866f-19dc8c78b965 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:28.937440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804ad1a-9225-4e37-866f-19dc8c78b965 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:28.937440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c804ad1a-9225-4e37-866f-19dc8c78b965 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:28.938826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:28.943575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16d273-71fb-4559-a222-82d6fe3c5f11 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:28.943984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16d273-71fb-4559-a222-82d6fe3c5f11 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:28.944109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16d273-71fb-4559-a222-82d6fe3c5f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:28.964037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7e16d273-71fb-4559-a222-82d6fe3c5f11 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:28.965360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7e16d273-71fb-4559-a222-82d6fe3c5f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:28.974095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:24:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:24:29.127120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c804ad1a-9225-4e37-866f-19dc8c78b965 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:29.141228 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c804ad1a-9225-4e37-866f-19dc8c78b965 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:29.147098 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45935b1-966d-42b0-ac5b-8b54e6684bf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:29.147690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45935b1-966d-42b0-ac5b-8b54e6684bf2 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:29.147973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45935b1-966d-42b0-ac5b-8b54e6684bf2 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:29.148243 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45935b1-966d-42b0-ac5b-8b54e6684bf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:29.157191 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45935b1-966d-42b0-ac5b-8b54e6684bf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:29.158867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45935b1-966d-42b0-ac5b-8b54e6684bf2 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:29.171224 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 760/1516] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:24:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:24:30.430549 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fe5b6-4804-4b6c-9728-444165d79150 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:30.431119 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fe5b6-4804-4b6c-9728-444165d79150 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:30.431119 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9fe5b6-4804-4b6c-9728-444165d79150 None None] PATH_INFO: `/v3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:30.432400 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 757/1517] 10.222.0.15 () {54 vars in 925 bytes} [Wed Aug 24 15:24:30 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:24:30.438663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99116897-a70b-4a24-9e0a-76e074cd4b01 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:30.439106 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99116897-a70b-4a24-9e0a-76e074cd4b01 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:30.439211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99116897-a70b-4a24-9e0a-76e074cd4b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:30.498496 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-99116897-a70b-4a24-9e0a-76e074cd4b01 None None] MFA Rules not processed for user `fb745b98e3ae4723bad44428646f83e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:30.499969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-99116897-a70b-4a24-9e0a-76e074cd4b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:30.517842 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:30.575938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 761/1518] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Aug 24 15:24:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:24:34.487708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3a38e-c8fd-4352-b478-bb140c9c9d58 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:34.488269 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3a38e-c8fd-4352-b478-bb140c9c9d58 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:34.488269 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3a38e-c8fd-4352-b478-bb140c9c9d58 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:34.489490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 758/1519] 10.222.0.76 () {54 vars in 961 bytes} [Wed Aug 24 15:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:34.494658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec798c76-0e39-4129-82d6-b49dbf2c2223 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:34.495044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec798c76-0e39-4129-82d6-b49dbf2c2223 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:34.495208 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec798c76-0e39-4129-82d6-b49dbf2c2223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:34.514953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ec798c76-0e39-4129-82d6-b49dbf2c2223 None None] MFA Rules not processed for user `fb745b98e3ae4723bad44428646f83e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:34.516153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ec798c76-0e39-4129-82d6-b49dbf2c2223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:34.524898 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 762/1520] 10.222.0.76 () {58 vars in 1058 bytes} [Wed Aug 24 15:24:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:24:34.533267 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3a38e-c8fd-4352-b478-bb140c9c9d58 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:34.547943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3a38e-c8fd-4352-b478-bb140c9c9d58 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:34.553408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e1351-9c98-4b25-aa41-ec4aa5e38496 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:34.553996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e1351-9c98-4b25-aa41-ec4aa5e38496 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:34.554304 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e1351-9c98-4b25-aa41-ec4aa5e38496 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:34.554578 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e1351-9c98-4b25-aa41-ec4aa5e38496 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:34.563194 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e1351-9c98-4b25-aa41-ec4aa5e38496 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:34.564761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e1351-9c98-4b25-aa41-ec4aa5e38496 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:34.574284 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 759/1521] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:24:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:24:40.399538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d27560-3197-4cf7-a027-bd31f1277325 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:40.400108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d27560-3197-4cf7-a027-bd31f1277325 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:40.400108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d27560-3197-4cf7-a027-bd31f1277325 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:40.401738 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:40.407797 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535965ce-5efb-4b5c-b1de-32f95a476150 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:40.408287 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535965ce-5efb-4b5c-b1de-32f95a476150 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:40.408621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535965ce-5efb-4b5c-b1de-32f95a476150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:40.431496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-535965ce-5efb-4b5c-b1de-32f95a476150 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:40.433452 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-535965ce-5efb-4b5c-b1de-32f95a476150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:40.446479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:24:40.603445 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d27560-3197-4cf7-a027-bd31f1277325 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:40.619504 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d27560-3197-4cf7-a027-bd31f1277325 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:40.625760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10db27cb-018d-4b2d-ba0b-e5df31124d18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:40.626430 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db27cb-018d-4b2d-ba0b-e5df31124d18 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:40.626705 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db27cb-018d-4b2d-ba0b-e5df31124d18 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:40.626973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10db27cb-018d-4b2d-ba0b-e5df31124d18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:40.635747 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db27cb-018d-4b2d-ba0b-e5df31124d18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:40.637442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10db27cb-018d-4b2d-ba0b-e5df31124d18 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:40.647034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 764/1524] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:24:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:24:41.472062 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d858208-9446-4b7e-bc81-6b13425b1d51 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:41.472915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d858208-9446-4b7e-bc81-6b13425b1d51 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:41.472915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d858208-9446-4b7e-bc81-6b13425b1d51 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:41.474042 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 761/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 15:24:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:41.479794 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4191d1-84c2-4376-ac2f-d1afd7e0652f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:41.480302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4191d1-84c2-4376-ac2f-d1afd7e0652f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:41.480422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4191d1-84c2-4376-ac2f-d1afd7e0652f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:41.508582 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ba4191d1-84c2-4376-ac2f-d1afd7e0652f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:41.509928 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ba4191d1-84c2-4376-ac2f-d1afd7e0652f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:41.519249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 765/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 15:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:24:41.676864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d858208-9446-4b7e-bc81-6b13425b1d51 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:41.692558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d858208-9446-4b7e-bc81-6b13425b1d51 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:41.701797 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d3777c-3a2e-4fbf-9eb6-3a43a7fc93fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:41.702452 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3777c-3a2e-4fbf-9eb6-3a43a7fc93fa service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:41.702735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3777c-3a2e-4fbf-9eb6-3a43a7fc93fa service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:41.703008 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d3777c-3a2e-4fbf-9eb6-3a43a7fc93fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:41.712830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3777c-3a2e-4fbf-9eb6-3a43a7fc93fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:41.714571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d3777c-3a2e-4fbf-9eb6-3a43a7fc93fa service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:41.725378 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 762/1527] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:24:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:24:45.581715 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda6d5e-89cc-4ba0-b599-be50bbaf9afc None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:45.582626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda6d5e-89cc-4ba0-b599-be50bbaf9afc None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:45.582626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcda6d5e-89cc-4ba0-b599-be50bbaf9afc None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:45.585006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:24:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:45.592027 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac97d5-e074-40fc-ba82-81c5c985e6ef None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:45.592027 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac97d5-e074-40fc-ba82-81c5c985e6ef None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:45.592027 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac97d5-e074-40fc-ba82-81c5c985e6ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:45.610400 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-96ac97d5-e074-40fc-ba82-81c5c985e6ef None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:45.611600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-96ac97d5-e074-40fc-ba82-81c5c985e6ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:45.620307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:24:45.906932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda6d5e-89cc-4ba0-b599-be50bbaf9afc None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:45.909153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bcda6d5e-89cc-4ba0-b599-be50bbaf9afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:45.989513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcda6d5e-89cc-4ba0-b599-be50bbaf9afc None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:45.993237 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8447a4e3-d2a6-4482-90bb-2d9fef19e2a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:45.993839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447a4e3-d2a6-4482-90bb-2d9fef19e2a2 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:45.994143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447a4e3-d2a6-4482-90bb-2d9fef19e2a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:45.994413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8447a4e3-d2a6-4482-90bb-2d9fef19e2a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:46.003737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447a4e3-d2a6-4482-90bb-2d9fef19e2a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:46.005169 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8447a4e3-d2a6-4482-90bb-2d9fef19e2a2 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:46.014893 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 767/1530] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:24:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:24:47.948742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c8cea-b1c1-412e-b107-f14f8b8d18b8 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:47.949312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c8cea-b1c1-412e-b107-f14f8b8d18b8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:47.949312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576c8cea-b1c1-412e-b107-f14f8b8d18b8 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:47.950683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:47.956261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3efd01-dee5-456e-bbd5-06f9d04ebcdd None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:47.956675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3efd01-dee5-456e-bbd5-06f9d04ebcdd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:47.956819 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3efd01-dee5-456e-bbd5-06f9d04ebcdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:47.976536 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ae3efd01-dee5-456e-bbd5-06f9d04ebcdd None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:47.977745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ae3efd01-dee5-456e-bbd5-06f9d04ebcdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:47.985708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:24:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:24:48.277194 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c8cea-b1c1-412e-b107-f14f8b8d18b8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:48.289240 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576c8cea-b1c1-412e-b107-f14f8b8d18b8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:48.292816 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ffcd5-c062-4884-99d9-1e877485db72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:48.293410 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ffcd5-c062-4884-99d9-1e877485db72 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:48.293690 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ffcd5-c062-4884-99d9-1e877485db72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:48.293960 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ffcd5-c062-4884-99d9-1e877485db72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:48.303488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ffcd5-c062-4884-99d9-1e877485db72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:48.304900 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ffcd5-c062-4884-99d9-1e877485db72 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:48.314292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 765/1533] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:24:51.772074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ecdf7-e0f7-4852-98b8-ab198c18d900 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:51.772617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ecdf7-e0f7-4852-98b8-ab198c18d900 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:51.772617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2ecdf7-e0f7-4852-98b8-ab198c18d900 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:51.773957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:24:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:24:51.779384 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b47d4-e31e-4028-b009-86d7262fc239 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:51.779829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b47d4-e31e-4028-b009-86d7262fc239 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:51.779964 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255b47d4-e31e-4028-b009-86d7262fc239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:51.801295 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-255b47d4-e31e-4028-b009-86d7262fc239 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:51.802448 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-255b47d4-e31e-4028-b009-86d7262fc239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:51.810370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:24:52.361794 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ecdf7-e0f7-4852-98b8-ab198c18d900 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:52.363543 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4a2ecdf7-e0f7-4852-98b8-ab198c18d900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:52.446685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:52.462016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2ecdf7-e0f7-4852-98b8-ab198c18d900 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:52.470526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ccacf4-1503-420e-9550-9f364c5ed35a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:52.471676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccacf4-1503-420e-9550-9f364c5ed35a service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:52.471956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccacf4-1503-420e-9550-9f364c5ed35a service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:52.472220 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ccacf4-1503-420e-9550-9f364c5ed35a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:52.481494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ccacf4-1503-420e-9550-9f364c5ed35a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:52.482960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ccacf4-1503-420e-9550-9f364c5ed35a service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:52.492417 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 770/1536] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:24:54.437439 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bbb95-22be-40d5-9d76-947d7a410c2c None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:54.438024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bbb95-22be-40d5-9d76-947d7a410c2c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:54.438024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bbb95-22be-40d5-9d76-947d7a410c2c None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:54.439352 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 15:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:24:54.444755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610767f-0d83-453e-9550-e65e496c1d24 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:54.445223 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610767f-0d83-453e-9550-e65e496c1d24 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:54.445334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0610767f-0d83-453e-9550-e65e496c1d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:54.465573 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0610767f-0d83-453e-9550-e65e496c1d24 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:24:54.466825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0610767f-0d83-453e-9550-e65e496c1d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:54.475965 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 15:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:24:55.033934 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bbb95-22be-40d5-9d76-947d7a410c2c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:24:55.035739 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e50bbb95-22be-40d5-9d76-947d7a410c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:24:55.043653 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:24:55.051345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bbb95-22be-40d5-9d76-947d7a410c2c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:24:55.055063 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e88375-a825-4359-89b2-336a49556fd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:24:55.055688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e88375-a825-4359-89b2-336a49556fd1 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:24:55.055963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e88375-a825-4359-89b2-336a49556fd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:24:55.056263 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e88375-a825-4359-89b2-336a49556fd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:24:55.065528 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e88375-a825-4359-89b2-336a49556fd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:24:55.067174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e88375-a825-4359-89b2-336a49556fd1 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:24:55.077810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 768/1539] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:25:04.460531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=11 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:25:05.072876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:20.190459 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bbd7a-3b5b-4145-a459-fe54d58293ee None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.191149 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bbd7a-3b5b-4145-a459-fe54d58293ee None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.191149 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bbd7a-3b5b-4145-a459-fe54d58293ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.220682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-807bbd7a-3b5b-4145-a459-fe54d58293ee None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:20.222503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-807bbd7a-3b5b-4145-a459-fe54d58293ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:20.244299 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 772/1540] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:20.253660 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0ce15-84c0-4663-a85f-e94b7f2edd1b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.254393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0ce15-84c0-4663-a85f-e94b7f2edd1b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.254393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a0ce15-84c0-4663-a85f-e94b7f2edd1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.283530 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-86a0ce15-84c0-4663-a85f-e94b7f2edd1b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:20.284764 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-86a0ce15-84c0-4663-a85f-e94b7f2edd1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:20.304481 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 769/1541] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:20.309455 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bbd7a-3b5b-4145-a459-fe54d58293ee None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:20.324725 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bbd7a-3b5b-4145-a459-fe54d58293ee None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:20.338255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f058e5a-b538-4283-911e-b061fedd1fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:20.338255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f058e5a-b538-4283-911e-b061fedd1fb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.338255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f058e5a-b538-4283-911e-b061fedd1fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.338255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f058e5a-b538-4283-911e-b061fedd1fb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.338750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f058e5a-b538-4283-911e-b061fedd1fb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:20.338750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f058e5a-b538-4283-911e-b061fedd1fb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:20.338750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f058e5a-b538-4283-911e-b061fedd1fb3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:20.359802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 773/1542] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:20.368856 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0ce15-84c0-4663-a85f-e94b7f2edd1b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:20.388204 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a0ce15-84c0-4663-a85f-e94b7f2edd1b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:20.397783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a51882-5919-48c1-a0ea-1985413ba09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:20.398464 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a51882-5919-48c1-a0ea-1985413ba09c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.398733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a51882-5919-48c1-a0ea-1985413ba09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.399232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a51882-5919-48c1-a0ea-1985413ba09c admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.399884 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a51882-5919-48c1-a0ea-1985413ba09c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:20.401848 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a51882-5919-48c1-a0ea-1985413ba09c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:20.456185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 770/1543] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:26:20 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:20.457841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568683fb-e329-4748-8b4b-4dbaefd11c2f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.458336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568683fb-e329-4748-8b4b-4dbaefd11c2f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.458610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568683fb-e329-4748-8b4b-4dbaefd11c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.472483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a51882-5919-48c1-a0ea-1985413ba09c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:20.490646 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-568683fb-e329-4748-8b4b-4dbaefd11c2f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:20.492349 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-568683fb-e329-4748-8b4b-4dbaefd11c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:20.575005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a51882-5919-48c1-a0ea-1985413ba09c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:20.592977 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b8252-3c84-4759-8d49-7e3e88d8b736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:20.594170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b8252-3c84-4759-8d49-7e3e88d8b736 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.594534 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b8252-3c84-4759-8d49-7e3e88d8b736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.594993 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b8252-3c84-4759-8d49-7e3e88d8b736 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.595636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b8252-3c84-4759-8d49-7e3e88d8b736 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:20.596674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-579b8252-3c84-4759-8d49-7e3e88d8b736 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:26:20.596674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 24 15:26:20.596674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: Aug 24 15:26:20.597899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b8252-3c84-4759-8d49-7e3e88d8b736 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:20.641877 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 771/1544] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:26:20 2022] POST /identity/v3/users => generated 287 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:20.649706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b8252-3c84-4759-8d49-7e3e88d8b736 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:20.664938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 774/1545] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:20.671468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b8252-3c84-4759-8d49-7e3e88d8b736 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:20.676795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83535d-132b-42fa-b3b9-a9891f925dac None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.677115 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83535d-132b-42fa-b3b9-a9891f925dac None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.677351 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83535d-132b-42fa-b3b9-a9891f925dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.681281 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bcaff-7f6d-474d-88e3-14733dd375b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:20.681911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bcaff-7f6d-474d-88e3-14733dd375b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.682185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bcaff-7f6d-474d-88e3-14733dd375b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.682474 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863bcaff-7f6d-474d-88e3-14733dd375b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.683124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bcaff-7f6d-474d-88e3-14733dd375b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:20.684719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863bcaff-7f6d-474d-88e3-14733dd375b4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:20.704794 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 772/1546] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:26:20.710011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9d83535d-132b-42fa-b3b9-a9891f925dac None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:20.713703 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bcaff-7f6d-474d-88e3-14733dd375b4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:20.715840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9d83535d-132b-42fa-b3b9-a9891f925dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:20.728549 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863bcaff-7f6d-474d-88e3-14733dd375b4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:20.733794 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 775/1547] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:20.735752 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51638c3-20bf-4e03-96dd-7df8c762e9f4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.736149 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51638c3-20bf-4e03-96dd-7df8c762e9f4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.736430 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51638c3-20bf-4e03-96dd-7df8c762e9f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.740565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de308f9-12eb-4d69-aa6d-99268de0591b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:20.741189 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de308f9-12eb-4d69-aa6d-99268de0591b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.741467 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de308f9-12eb-4d69-aa6d-99268de0591b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.741736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de308f9-12eb-4d69-aa6d-99268de0591b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.742420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de308f9-12eb-4d69-aa6d-99268de0591b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:20.744236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de308f9-12eb-4d69-aa6d-99268de0591b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:20.762180 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 773/1548] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:26:20.763074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de308f9-12eb-4d69-aa6d-99268de0591b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:20.776006 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de308f9-12eb-4d69-aa6d-99268de0591b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:20.783515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a51638c3-20bf-4e03-96dd-7df8c762e9f4 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:20.785271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a51638c3-20bf-4e03-96dd-7df8c762e9f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:20.791048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38fa1f-42bf-4d35-876b-77eb971c2553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:20.791686 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38fa1f-42bf-4d35-876b-77eb971c2553 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.791960 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38fa1f-42bf-4d35-876b-77eb971c2553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.792221 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38fa1f-42bf-4d35-876b-77eb971c2553 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.792869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38fa1f-42bf-4d35-876b-77eb971c2553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:20.793158 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38fa1f-42bf-4d35-876b-77eb971c2553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:20.794157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-cf38fa1f-42bf-4d35-876b-77eb971c2553 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:26:20.794157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 24 15:26:20.794157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: Aug 24 15:26:20.795065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38fa1f-42bf-4d35-876b-77eb971c2553 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:20.804734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 776/1549] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:20.806162 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fa186-c540-4910-b164-179602506afb None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.806488 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fa186-c540-4910-b164-179602506afb None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.806762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80fa186-c540-4910-b164-179602506afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.814015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 774/1550] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:20.814864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38fa1f-42bf-4d35-876b-77eb971c2553 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:20.830818 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38fa1f-42bf-4d35-876b-77eb971c2553 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:20.838312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b80fa186-c540-4910-b164-179602506afb None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:20.839788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b80fa186-c540-4910-b164-179602506afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:20.841017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6a61d9-2194-4058-9910-d1b37f5b7802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:20.841803 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a61d9-2194-4058-9910-d1b37f5b7802 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.842200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a61d9-2194-4058-9910-d1b37f5b7802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.842719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6a61d9-2194-4058-9910-d1b37f5b7802 admin admin] PATH_INFO: `/v3/projects/c237131ebfac4007b296b5dd564e89e5/users/5579431178d0499188de35c56b925a63/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.852107 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 777/1551] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:20.853373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f89ddd-6924-40b8-bfaa-de52747b2e08 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.853665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f89ddd-6924-40b8-bfaa-de52747b2e08 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.853933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f89ddd-6924-40b8-bfaa-de52747b2e08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.868025 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a61d9-2194-4058-9910-d1b37f5b7802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c237131ebfac4007b296b5dd564e89e5, user_id=5579431178d0499188de35c56b925a63, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:20.871179 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6a61d9-2194-4058-9910-d1b37f5b7802 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:20.877109 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-71f89ddd-6924-40b8-bfaa-de52747b2e08 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:20.878588 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-71f89ddd-6924-40b8-bfaa-de52747b2e08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:20.886854 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 775/1552] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:20 2022] PUT /identity/v3/projects/c237131ebfac4007b296b5dd564e89e5/users/5579431178d0499188de35c56b925a63/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:20.888253 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bff5c1-ad28-4d09-a479-a158876980e5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.889011 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bff5c1-ad28-4d09-a479-a158876980e5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.889011 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bff5c1-ad28-4d09-a479-a158876980e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.911888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d1bff5c1-ad28-4d09-a479-a158876980e5 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:20.911888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d1bff5c1-ad28-4d09-a479-a158876980e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:20.947664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 778/1553] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:20.948502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f89ddd-6924-40b8-bfaa-de52747b2e08 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:20.957980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f89ddd-6924-40b8-bfaa-de52747b2e08 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:20.963134 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9d2796-822c-4633-a5ec-aee58e58be50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:20.963683 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d2796-822c-4633-a5ec-aee58e58be50 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.963962 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d2796-822c-4633-a5ec-aee58e58be50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.964231 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9d2796-822c-4633-a5ec-aee58e58be50 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.964858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d2796-822c-4633-a5ec-aee58e58be50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:20.965003 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:20.965915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: WARNING py.warnings [None req-5b9d2796-822c-4633-a5ec-aee58e58be50 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:26:20.965915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: warnings.warn(msg) Aug 24 15:26:20.965915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: Aug 24 15:26:20.967477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9d2796-822c-4633-a5ec-aee58e58be50 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:20.979615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 776/1554] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:20.980941 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690caab-3a18-448f-aee1-3b9dd455a731 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:20.981290 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690caab-3a18-448f-aee1-3b9dd455a731 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:20.981722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d690caab-3a18-448f-aee1-3b9dd455a731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:20.998952 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 779/1555] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:26:20 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:21.000421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a71af-7b50-4136-9bbf-3ebbe6a33137 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.000768 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a71af-7b50-4136-9bbf-3ebbe6a33137 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.001108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555a71af-7b50-4136-9bbf-3ebbe6a33137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.003455 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d690caab-3a18-448f-aee1-3b9dd455a731 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:21.004662 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d690caab-3a18-448f-aee1-3b9dd455a731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:21.021247 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-555a71af-7b50-4136-9bbf-3ebbe6a33137 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:21.022957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-555a71af-7b50-4136-9bbf-3ebbe6a33137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:21.059304 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:21.078666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 777/1556] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:21.079933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031029dd-8860-4385-bf2b-6de77519cb28 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.080224 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031029dd-8860-4385-bf2b-6de77519cb28 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.080500 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031029dd-8860-4385-bf2b-6de77519cb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.087719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 780/1557] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:21.089120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1c18b-b1d5-4ddc-978d-f9509b1a886b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.089411 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1c18b-b1d5-4ddc-978d-f9509b1a886b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.089683 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d1c18b-b1d5-4ddc-978d-f9509b1a886b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.106966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-031029dd-8860-4385-bf2b-6de77519cb28 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:21.108292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-031029dd-8860-4385-bf2b-6de77519cb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:21.117665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-88d1c18b-b1d5-4ddc-978d-f9509b1a886b None None] MFA Rules not processed for user `5579431178d0499188de35c56b925a63`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:21.118053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 778/1558] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:21.118930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-88d1c18b-b1d5-4ddc-978d-f9509b1a886b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:21.119287 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cf9a7-5beb-43ba-b5be-ef356fe91870 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.119567 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cf9a7-5beb-43ba-b5be-ef356fe91870 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.119832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cf9a7-5beb-43ba-b5be-ef356fe91870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.139211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a97cf9a7-5beb-43ba-b5be-ef356fe91870 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:21.140459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a97cf9a7-5beb-43ba-b5be-ef356fe91870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:21.150282 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 779/1559] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:21.150282 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47194fb0-00ab-4c70-8920-e432ad3eb572 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.150282 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47194fb0-00ab-4c70-8920-e432ad3eb572 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.150633 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47194fb0-00ab-4c70-8920-e432ad3eb572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.170052 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-47194fb0-00ab-4c70-8920-e432ad3eb572 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:21.171310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-47194fb0-00ab-4c70-8920-e432ad3eb572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:21.180059 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 780/1560] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:21.181304 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af68e2a-2420-46bf-9bbd-8af374c5d86e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.181587 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af68e2a-2420-46bf-9bbd-8af374c5d86e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.181859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af68e2a-2420-46bf-9bbd-8af374c5d86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.194669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 781/1561] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:21.195490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d1c18b-b1d5-4ddc-978d-f9509b1a886b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.201729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8af68e2a-2420-46bf-9bbd-8af374c5d86e None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:21.202988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8af68e2a-2420-46bf-9bbd-8af374c5d86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:21.206037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d1c18b-b1d5-4ddc-978d-f9509b1a886b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.211051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d174a-573a-441a-ba5a-20d770d5ecd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.211241 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 781/1562] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:21.211446 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d174a-573a-441a-ba5a-20d770d5ecd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.211702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d174a-573a-441a-ba5a-20d770d5ecd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.211824 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af68e2a-2420-46bf-9bbd-8af374c5d86e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.211993 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d174a-573a-441a-ba5a-20d770d5ecd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.212572 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d174a-573a-441a-ba5a-20d770d5ecd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.214694 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d174a-573a-441a-ba5a-20d770d5ecd4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.221269 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af68e2a-2420-46bf-9bbd-8af374c5d86e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.225989 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a04681-ccb5-4248-91c6-bca6db525fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.226656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a04681-ccb5-4248-91c6-bca6db525fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.227002 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a04681-ccb5-4248-91c6-bca6db525fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.227275 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a04681-ccb5-4248-91c6-bca6db525fd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.228016 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a04681-ccb5-4248-91c6-bca6db525fd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:21.228310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a04681-ccb5-4248-91c6-bca6db525fd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.230109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a04681-ccb5-4248-91c6-bca6db525fd3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.238360 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 782/1563] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/users => generated 291 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:21.239776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85300146-28a3-45fc-8547-1cbd073c8f32 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.240066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85300146-28a3-45fc-8547-1cbd073c8f32 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.240336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85300146-28a3-45fc-8547-1cbd073c8f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.246509 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 782/1564] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:21.247351 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a04681-ccb5-4248-91c6-bca6db525fd3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.261272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-85300146-28a3-45fc-8547-1cbd073c8f32 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:21.262522 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-85300146-28a3-45fc-8547-1cbd073c8f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:21.264262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a04681-ccb5-4248-91c6-bca6db525fd3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.269156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51ba21-1d39-49b8-b6d3-8263de58edd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.269746 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51ba21-1d39-49b8-b6d3-8263de58edd1 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.270021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51ba21-1d39-49b8-b6d3-8263de58edd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.270324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d51ba21-1d39-49b8-b6d3-8263de58edd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.270556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 783/1565] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:21.271878 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7a8c7-57e6-4948-af3e-d5ef5d04d647 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.272163 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7a8c7-57e6-4948-af3e-d5ef5d04d647 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.272437 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea7a8c7-57e6-4948-af3e-d5ef5d04d647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.274535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51ba21-1d39-49b8-b6d3-8263de58edd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:21.274854 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51ba21-1d39-49b8-b6d3-8263de58edd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.276488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d51ba21-1d39-49b8-b6d3-8263de58edd1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.296343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 783/1566] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:21.296343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51ba21-1d39-49b8-b6d3-8263de58edd1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.298164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-fea7a8c7-57e6-4948-af3e-d5ef5d04d647 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:21.299400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fea7a8c7-57e6-4948-af3e-d5ef5d04d647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:21.300341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d51ba21-1d39-49b8-b6d3-8263de58edd1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.305384 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d03b9c-9e60-478d-a1f4-13cf58e355f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.305968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d03b9c-9e60-478d-a1f4-13cf58e355f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.306262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d03b9c-9e60-478d-a1f4-13cf58e355f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.306561 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d03b9c-9e60-478d-a1f4-13cf58e355f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.307189 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d03b9c-9e60-478d-a1f4-13cf58e355f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:21.307485 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d03b9c-9e60-478d-a1f4-13cf58e355f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.308077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 784/1567] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:21.311872 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7a8c7-57e6-4948-af3e-d5ef5d04d647 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.311981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d03b9c-9e60-478d-a1f4-13cf58e355f1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.319448 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea7a8c7-57e6-4948-af3e-d5ef5d04d647 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.321756 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 784/1568] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:21.322988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d03b9c-9e60-478d-a1f4-13cf58e355f1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.329762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ac72a-297b-458b-ac5d-b269cf897403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.330406 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ac72a-297b-458b-ac5d-b269cf897403 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.330682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ac72a-297b-458b-ac5d-b269cf897403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.330949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ac72a-297b-458b-ac5d-b269cf897403 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.331569 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ac72a-297b-458b-ac5d-b269cf897403 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.333473 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ac72a-297b-458b-ac5d-b269cf897403 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.334119 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d03b9c-9e60-478d-a1f4-13cf58e355f1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.338965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be42ed-c8c8-4182-bf26-642d183ee02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.339306 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be42ed-c8c8-4182-bf26-642d183ee02b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.339585 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be42ed-c8c8-4182-bf26-642d183ee02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.339853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be42ed-c8c8-4182-bf26-642d183ee02b admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.340485 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be42ed-c8c8-4182-bf26-642d183ee02b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:21.340783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be42ed-c8c8-4182-bf26-642d183ee02b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.342365 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71be42ed-c8c8-4182-bf26-642d183ee02b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.355414 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 785/1569] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:21.356238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be42ed-c8c8-4182-bf26-642d183ee02b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.361165 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 785/1570] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:21.362354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ac72a-297b-458b-ac5d-b269cf897403 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.366931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71be42ed-c8c8-4182-bf26-642d183ee02b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.424898 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ac72a-297b-458b-ac5d-b269cf897403 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.425206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760588e8-b991-47c1-a65f-b97c9fa8ad39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.425817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760588e8-b991-47c1-a65f-b97c9fa8ad39 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.426091 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760588e8-b991-47c1-a65f-b97c9fa8ad39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.426392 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760588e8-b991-47c1-a65f-b97c9fa8ad39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.427016 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760588e8-b991-47c1-a65f-b97c9fa8ad39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.428597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760588e8-b991-47c1-a65f-b97c9fa8ad39 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.429578 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab391679-0a24-4eba-b31c-6d048b83b31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.430188 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab391679-0a24-4eba-b31c-6d048b83b31c admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.430496 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab391679-0a24-4eba-b31c-6d048b83b31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.430773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab391679-0a24-4eba-b31c-6d048b83b31c admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.431394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab391679-0a24-4eba-b31c-6d048b83b31c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.433284 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab391679-0a24-4eba-b31c-6d048b83b31c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.438969 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 786/1571] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:21.439763 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760588e8-b991-47c1-a65f-b97c9fa8ad39 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.448896 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760588e8-b991-47c1-a65f-b97c9fa8ad39 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.454165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143f368-1b28-465e-b15a-055a8c133001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.454816 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143f368-1b28-465e-b15a-055a8c133001 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.455233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143f368-1b28-465e-b15a-055a8c133001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.455504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4143f368-1b28-465e-b15a-055a8c133001 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.456181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143f368-1b28-465e-b15a-055a8c133001 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:21.456461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143f368-1b28-465e-b15a-055a8c133001 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.458039 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4143f368-1b28-465e-b15a-055a8c133001 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.464755 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:21.465224 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 786/1572] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:21.469264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab391679-0a24-4eba-b31c-6d048b83b31c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.471240 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 787/1573] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:21.472106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143f368-1b28-465e-b15a-055a8c133001 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.473637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:21.488238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4143f368-1b28-465e-b15a-055a8c133001 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.543416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab391679-0a24-4eba-b31c-6d048b83b31c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.548371 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7832c9c-1263-4514-be3f-9592c7741c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.548977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7832c9c-1263-4514-be3f-9592c7741c91 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.549261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7832c9c-1263-4514-be3f-9592c7741c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.549595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7832c9c-1263-4514-be3f-9592c7741c91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.550248 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7832c9c-1263-4514-be3f-9592c7741c91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.552172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7832c9c-1263-4514-be3f-9592c7741c91 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.557073 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa29057-616d-45ee-9e6a-f9f0113ae20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.557750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa29057-616d-45ee-9e6a-f9f0113ae20a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.558027 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa29057-616d-45ee-9e6a-f9f0113ae20a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.558329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa29057-616d-45ee-9e6a-f9f0113ae20a admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.558976 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa29057-616d-45ee-9e6a-f9f0113ae20a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:21.559296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa29057-616d-45ee-9e6a-f9f0113ae20a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.560890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa29057-616d-45ee-9e6a-f9f0113ae20a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.573830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 788/1574] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:21.575055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa29057-616d-45ee-9e6a-f9f0113ae20a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.586245 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa29057-616d-45ee-9e6a-f9f0113ae20a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.588324 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 787/1575] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/projects => generated 389 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:21.588934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7832c9c-1263-4514-be3f-9592c7741c91 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.650303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046b390-db1d-4825-a746-41de32a093eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.650922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046b390-db1d-4825-a746-41de32a093eb admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.651206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046b390-db1d-4825-a746-41de32a093eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.651479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046b390-db1d-4825-a746-41de32a093eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.652112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046b390-db1d-4825-a746-41de32a093eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.654025 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7046b390-db1d-4825-a746-41de32a093eb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.655072 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7832c9c-1263-4514-be3f-9592c7741c91 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.660090 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015520c8-08a9-40a1-9646-cde9733a7560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.660697 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015520c8-08a9-40a1-9646-cde9733a7560 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.660980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015520c8-08a9-40a1-9646-cde9733a7560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.661280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015520c8-08a9-40a1-9646-cde9733a7560 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.662023 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015520c8-08a9-40a1-9646-cde9733a7560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.663982 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015520c8-08a9-40a1-9646-cde9733a7560 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.688669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 789/1576] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:21.688669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046b390-db1d-4825-a746-41de32a093eb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.693475 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 788/1577] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:21.693866 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015520c8-08a9-40a1-9646-cde9733a7560 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.706586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015520c8-08a9-40a1-9646-cde9733a7560 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.754883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046b390-db1d-4825-a746-41de32a093eb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.759262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77178d1e-8496-4b0d-8890-02dd69389a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.759878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77178d1e-8496-4b0d-8890-02dd69389a5f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.760165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77178d1e-8496-4b0d-8890-02dd69389a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.760580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77178d1e-8496-4b0d-8890-02dd69389a5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.761227 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77178d1e-8496-4b0d-8890-02dd69389a5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.763359 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77178d1e-8496-4b0d-8890-02dd69389a5f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.772988 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707ff49-c624-42d9-b1bf-18d8142d5318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.773717 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707ff49-c624-42d9-b1bf-18d8142d5318 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.773933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707ff49-c624-42d9-b1bf-18d8142d5318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.774197 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707ff49-c624-42d9-b1bf-18d8142d5318 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.774866 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707ff49-c624-42d9-b1bf-18d8142d5318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.776461 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b707ff49-c624-42d9-b1bf-18d8142d5318 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.787405 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 789/1578] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:21.788152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707ff49-c624-42d9-b1bf-18d8142d5318 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.794347 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 790/1579] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:21.795215 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77178d1e-8496-4b0d-8890-02dd69389a5f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.800010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:21.801957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707ff49-c624-42d9-b1bf-18d8142d5318 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.813859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:21.864908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77178d1e-8496-4b0d-8890-02dd69389a5f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.868272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb9f12-de57-4c5e-af90-77f0242f04d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.868885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb9f12-de57-4c5e-af90-77f0242f04d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.869164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb9f12-de57-4c5e-af90-77f0242f04d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.869430 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eb9f12-de57-4c5e-af90-77f0242f04d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.870050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb9f12-de57-4c5e-af90-77f0242f04d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.870349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ec21a-32f8-4e33-9881-fcc4717dee61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.871059 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ec21a-32f8-4e33-9881-fcc4717dee61 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.871435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ec21a-32f8-4e33-9881-fcc4717dee61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.871815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ec21a-32f8-4e33-9881-fcc4717dee61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.872003 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eb9f12-de57-4c5e-af90-77f0242f04d2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.872766 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ec21a-32f8-4e33-9881-fcc4717dee61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.875378 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ec21a-32f8-4e33-9881-fcc4717dee61 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.900465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 790/1580] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:21.901956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb9f12-de57-4c5e-af90-77f0242f04d2 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.916074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 791/1581] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:21.916305 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eb9f12-de57-4c5e-af90-77f0242f04d2 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.916418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ec21a-32f8-4e33-9881-fcc4717dee61 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:21.979791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ec21a-32f8-4e33-9881-fcc4717dee61 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:21.984883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb73cd5-e6bd-4653-8d77-86a7b846c3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.985154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889eaa11-7a78-4fc7-8446-339cd816c1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:21.985488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb73cd5-e6bd-4653-8d77-86a7b846c3c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.985616 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889eaa11-7a78-4fc7-8446-339cd816c1ac admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:21.985761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb73cd5-e6bd-4653-8d77-86a7b846c3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.985877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889eaa11-7a78-4fc7-8446-339cd816c1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:21.986023 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb73cd5-e6bd-4653-8d77-86a7b846c3c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.986125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889eaa11-7a78-4fc7-8446-339cd816c1ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:21.986673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb73cd5-e6bd-4653-8d77-86a7b846c3c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.986783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889eaa11-7a78-4fc7-8446-339cd816c1ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:21.988588 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb73cd5-e6bd-4653-8d77-86a7b846c3c0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:21.988711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889eaa11-7a78-4fc7-8446-339cd816c1ac admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.011889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 792/1582] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:22.012959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb73cd5-e6bd-4653-8d77-86a7b846c3c0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.025277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 791/1583] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:26:21 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:22.025277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889eaa11-7a78-4fc7-8446-339cd816c1ac admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.026120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb73cd5-e6bd-4653-8d77-86a7b846c3c0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.098124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74384b0d-4f6a-4738-9751-5ac30bb99564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.098761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74384b0d-4f6a-4738-9751-5ac30bb99564 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.099035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74384b0d-4f6a-4738-9751-5ac30bb99564 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.099303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74384b0d-4f6a-4738-9751-5ac30bb99564 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.099921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74384b0d-4f6a-4738-9751-5ac30bb99564 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.101822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74384b0d-4f6a-4738-9751-5ac30bb99564 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.103552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889eaa11-7a78-4fc7-8446-339cd816c1ac admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.108161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20503ecd-d1ab-4fd3-89b4-8521215dad4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.108764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20503ecd-d1ab-4fd3-89b4-8521215dad4b admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.109532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20503ecd-d1ab-4fd3-89b4-8521215dad4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.109947 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20503ecd-d1ab-4fd3-89b4-8521215dad4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.110492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20503ecd-d1ab-4fd3-89b4-8521215dad4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.112273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20503ecd-d1ab-4fd3-89b4-8521215dad4b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.127647 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 792/1584] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:22.127874 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 793/1585] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:26:22 2022] POST /identity/v3/users => generated 284 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:22.127973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20503ecd-d1ab-4fd3-89b4-8521215dad4b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.128077 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74384b0d-4f6a-4738-9751-5ac30bb99564 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.135637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20503ecd-d1ab-4fd3-89b4-8521215dad4b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.137309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74384b0d-4f6a-4738-9751-5ac30bb99564 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.142462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23fc51-6682-4c25-927a-381eece1744b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.143514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23fc51-6682-4c25-927a-381eece1744b admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.143938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23fc51-6682-4c25-927a-381eece1744b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.144062 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa0908-5b63-450c-863c-f768bf4e2194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.144374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23fc51-6682-4c25-927a-381eece1744b admin admin] PATH_INFO: `/v3/projects/63480381e3f44c7fbb93a62aa5b09800/users/05b1c0c6cdbc49929dc9f0cc14b9315c/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.144903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa0908-5b63-450c-863c-f768bf4e2194 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.145350 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa0908-5b63-450c-863c-f768bf4e2194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.145802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfa0908-5b63-450c-863c-f768bf4e2194 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.146837 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa0908-5b63-450c-863c-f768bf4e2194 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.149862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfa0908-5b63-450c-863c-f768bf4e2194 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.161829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:22.168884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23fc51-6682-4c25-927a-381eece1744b admin admin] RBAC: Authorizing `identity:create_grant(project_id=63480381e3f44c7fbb93a62aa5b09800, user_id=05b1c0c6cdbc49929dc9f0cc14b9315c, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.170191 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: WARNING py.warnings [None req-4d23fc51-6682-4c25-927a-381eece1744b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:26:22.170191 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: warnings.warn(msg) Aug 24 15:26:22.170191 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: Aug 24 15:26:22.173688 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d23fc51-6682-4c25-927a-381eece1744b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.179797 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 794/1586] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:26:22 2022] POST /identity/v3/users => generated 292 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:22.180785 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa0908-5b63-450c-863c-f768bf4e2194 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.185469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:22.193843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 793/1587] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:22 2022] PUT /identity/v3/projects/63480381e3f44c7fbb93a62aa5b09800/users/05b1c0c6cdbc49929dc9f0cc14b9315c/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:22.194986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23fc51-6682-4c25-927a-381eece1744b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.202145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfa0908-5b63-450c-863c-f768bf4e2194 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.266287 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd0c23-d0c3-4645-b375-6f7ffb656b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.267032 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd0c23-d0c3-4645-b375-6f7ffb656b66 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.267146 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd0c23-d0c3-4645-b375-6f7ffb656b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.267377 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fd0c23-d0c3-4645-b375-6f7ffb656b66 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.268023 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd0c23-d0c3-4645-b375-6f7ffb656b66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.269624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fd0c23-d0c3-4645-b375-6f7ffb656b66 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.269814 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d23fc51-6682-4c25-927a-381eece1744b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.274762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f4bb4-a75a-4f26-b851-95ea0e46e973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.275366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f4bb4-a75a-4f26-b851-95ea0e46e973 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.275641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f4bb4-a75a-4f26-b851-95ea0e46e973 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.275905 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f4bb4-a75a-4f26-b851-95ea0e46e973 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.276655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f4bb4-a75a-4f26-b851-95ea0e46e973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.278593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f4bb4-a75a-4f26-b851-95ea0e46e973 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.282364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 795/1588] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:22.286350 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd0c23-d0c3-4645-b375-6f7ffb656b66 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.299842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fd0c23-d0c3-4645-b375-6f7ffb656b66 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.302807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 794/1589] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:26:22 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:22.303611 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f4bb4-a75a-4f26-b851-95ea0e46e973 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.306075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f51740-f97e-458f-b422-83842c4c0713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.306731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f51740-f97e-458f-b422-83842c4c0713 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.307012 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f51740-f97e-458f-b422-83842c4c0713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.307281 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f51740-f97e-458f-b422-83842c4c0713 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.307925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f51740-f97e-458f-b422-83842c4c0713 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.309526 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f51740-f97e-458f-b422-83842c4c0713 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.314637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f4bb4-a75a-4f26-b851-95ea0e46e973 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.319472 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e923999-fa3f-4387-a92b-526064512f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.320087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e923999-fa3f-4387-a92b-526064512f96 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.320361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e923999-fa3f-4387-a92b-526064512f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.320869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 796/1590] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:22.320986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e923999-fa3f-4387-a92b-526064512f96 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.321301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e923999-fa3f-4387-a92b-526064512f96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.321484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f51740-f97e-458f-b422-83842c4c0713 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.323469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e923999-fa3f-4387-a92b-526064512f96 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.332060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f51740-f97e-458f-b422-83842c4c0713 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.336967 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1114e-1375-4ca1-8013-e6d04001e944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.337581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1114e-1375-4ca1-8013-e6d04001e944 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.337862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1114e-1375-4ca1-8013-e6d04001e944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.338134 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce1114e-1375-4ca1-8013-e6d04001e944 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.338899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1114e-1375-4ca1-8013-e6d04001e944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.340517 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce1114e-1375-4ca1-8013-e6d04001e944 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.347334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 795/1591] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:26:22 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:22.347334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e923999-fa3f-4387-a92b-526064512f96 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.355393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 797/1592] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:22.356424 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1114e-1375-4ca1-8013-e6d04001e944 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.364114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e923999-fa3f-4387-a92b-526064512f96 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.367802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce1114e-1375-4ca1-8013-e6d04001e944 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.369255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090a160-55e3-4339-a9f1-69c7f18bea18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.369990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090a160-55e3-4339-a9f1-69c7f18bea18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.370384 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090a160-55e3-4339-a9f1-69c7f18bea18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.370763 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3090a160-55e3-4339-a9f1-69c7f18bea18 admin admin] PATH_INFO: `/v3/projects/6a03c5044d764f21afc9b56b56e9a5d2/users/4be80290e6ab4299ba70931bb1cd8f03/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.372429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8dd2a-37fb-487b-9112-5cf493a5e7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.373024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8dd2a-37fb-487b-9112-5cf493a5e7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.373299 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8dd2a-37fb-487b-9112-5cf493a5e7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.373560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8dd2a-37fb-487b-9112-5cf493a5e7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.374192 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8dd2a-37fb-487b-9112-5cf493a5e7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.376317 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8dd2a-37fb-487b-9112-5cf493a5e7a5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.387017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 798/1593] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:22.387486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090a160-55e3-4339-a9f1-69c7f18bea18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a03c5044d764f21afc9b56b56e9a5d2, user_id=4be80290e6ab4299ba70931bb1cd8f03, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.388775 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991f5bd-b651-49cb-9957-34c465b00516 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.389242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991f5bd-b651-49cb-9957-34c465b00516 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.389661 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a991f5bd-b651-49cb-9957-34c465b00516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.391117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3090a160-55e3-4339-a9f1-69c7f18bea18 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.405749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 796/1594] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:22 2022] PUT /identity/v3/projects/6a03c5044d764f21afc9b56b56e9a5d2/users/4be80290e6ab4299ba70931bb1cd8f03/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:22.409539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090a160-55e3-4339-a9f1-69c7f18bea18 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.422806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a991f5bd-b651-49cb-9957-34c465b00516 None None] MFA Rules not processed for user `05b1c0c6cdbc49929dc9f0cc14b9315c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:22.424060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a991f5bd-b651-49cb-9957-34c465b00516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:22.482580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3090a160-55e3-4339-a9f1-69c7f18bea18 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.487411 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542121b-561d-42a9-84bb-118b2d92bba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.488089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542121b-561d-42a9-84bb-118b2d92bba8 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.488375 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542121b-561d-42a9-84bb-118b2d92bba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.488627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0542121b-561d-42a9-84bb-118b2d92bba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.489255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542121b-561d-42a9-84bb-118b2d92bba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.490914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0542121b-561d-42a9-84bb-118b2d92bba8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.501563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 797/1595] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:22.502450 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542121b-561d-42a9-84bb-118b2d92bba8 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.503981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 799/1596] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:22.505128 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991f5bd-b651-49cb-9957-34c465b00516 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.513463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0542121b-561d-42a9-84bb-118b2d92bba8 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.518898 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a991f5bd-b651-49cb-9957-34c465b00516 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.519119 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe690d1-7c5b-4c47-a5f0-80eb33719e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.519701 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe690d1-7c5b-4c47-a5f0-80eb33719e44 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.519980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe690d1-7c5b-4c47-a5f0-80eb33719e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.520241 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe690d1-7c5b-4c47-a5f0-80eb33719e44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.520884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe690d1-7c5b-4c47-a5f0-80eb33719e44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.522496 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe690d1-7c5b-4c47-a5f0-80eb33719e44 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.524690 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace2027-9e8c-4830-a48c-3df1a6c5f330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.525428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2027-9e8c-4830-a48c-3df1a6c5f330 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.525834 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2027-9e8c-4830-a48c-3df1a6c5f330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.526298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2027-9e8c-4830-a48c-3df1a6c5f330 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.527248 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace2027-9e8c-4830-a48c-3df1a6c5f330 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.529303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ace2027-9e8c-4830-a48c-3df1a6c5f330 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.536602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 798/1597] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:22.540945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe690d1-7c5b-4c47-a5f0-80eb33719e44 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.546277 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 800/1598] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:22.547649 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace2027-9e8c-4830-a48c-3df1a6c5f330 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.548190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe690d1-7c5b-4c47-a5f0-80eb33719e44 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.554998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fc7c3-dd11-4def-aa8b-ec7a21d96432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.555905 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fc7c3-dd11-4def-aa8b-ec7a21d96432 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.556191 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fc7c3-dd11-4def-aa8b-ec7a21d96432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.556466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0fc7c3-dd11-4def-aa8b-ec7a21d96432 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.557125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fc7c3-dd11-4def-aa8b-ec7a21d96432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.558985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0fc7c3-dd11-4def-aa8b-ec7a21d96432 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.561945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace2027-9e8c-4830-a48c-3df1a6c5f330 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.567521 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b58e9-3586-4264-ac31-9b0089580dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.568129 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b58e9-3586-4264-ac31-9b0089580dc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.568401 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b58e9-3586-4264-ac31-9b0089580dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.568665 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441b58e9-3586-4264-ac31-9b0089580dc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.569449 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b58e9-3586-4264-ac31-9b0089580dc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.570935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441b58e9-3586-4264-ac31-9b0089580dc8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.573695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 799/1599] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:22.574944 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fc7c3-dd11-4def-aa8b-ec7a21d96432 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.581490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 801/1600] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:22.582358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b58e9-3586-4264-ac31-9b0089580dc8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.588648 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0fc7c3-dd11-4def-aa8b-ec7a21d96432 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.593688 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae8b5d-bca7-4b14-910b-ac0a0c3b8b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.594417 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae8b5d-bca7-4b14-910b-ac0a0c3b8b48 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.594781 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae8b5d-bca7-4b14-910b-ac0a0c3b8b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.596568 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dae8b5d-bca7-4b14-910b-ac0a0c3b8b48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.596568 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae8b5d-bca7-4b14-910b-ac0a0c3b8b48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.597391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dae8b5d-bca7-4b14-910b-ac0a0c3b8b48 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.599779 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441b58e9-3586-4264-ac31-9b0089580dc8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.605232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5798f5-1723-4cd7-be38-43f89e5a33fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.605837 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5798f5-1723-4cd7-be38-43f89e5a33fa admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.606109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5798f5-1723-4cd7-be38-43f89e5a33fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.606405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5798f5-1723-4cd7-be38-43f89e5a33fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.607045 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5798f5-1723-4cd7-be38-43f89e5a33fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.608090 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 800/1601] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:22.608625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5798f5-1723-4cd7-be38-43f89e5a33fa admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.608909 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae8b5d-bca7-4b14-910b-ac0a0c3b8b48 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.619459 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dae8b5d-bca7-4b14-910b-ac0a0c3b8b48 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.622831 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 802/1602] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:22.623707 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5798f5-1723-4cd7-be38-43f89e5a33fa admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.628475 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d249465-492e-4792-aebf-92a1af154138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.629147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d249465-492e-4792-aebf-92a1af154138 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.629421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d249465-492e-4792-aebf-92a1af154138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.629688 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d249465-492e-4792-aebf-92a1af154138 admin admin] PATH_INFO: `/v3/projects/08285585867f4b89bf5548321022aa3e/users/be45cc3aa12e44a4b6d49976b6eff0b3/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.634372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5798f5-1723-4cd7-be38-43f89e5a33fa admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.638935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:22.639372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b84c4-f137-4d7d-b5bd-b300f15de1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.639997 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b84c4-f137-4d7d-b5bd-b300f15de1bd admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.640504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b84c4-f137-4d7d-b5bd-b300f15de1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.640664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16b84c4-f137-4d7d-b5bd-b300f15de1bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.641346 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b84c4-f137-4d7d-b5bd-b300f15de1bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.643544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16b84c4-f137-4d7d-b5bd-b300f15de1bd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.647690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d249465-492e-4792-aebf-92a1af154138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08285585867f4b89bf5548321022aa3e, user_id=be45cc3aa12e44a4b6d49976b6eff0b3, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.649872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:22.650731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d249465-492e-4792-aebf-92a1af154138 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.672164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 801/1603] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:22 2022] PUT /identity/v3/projects/08285585867f4b89bf5548321022aa3e/users/be45cc3aa12e44a4b6d49976b6eff0b3/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:22.672445 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d249465-492e-4792-aebf-92a1af154138 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.683751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 803/1604] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:26:22 2022] POST /identity/v3/projects => generated 381 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:22.684660 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b84c4-f137-4d7d-b5bd-b300f15de1bd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.742228 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d249465-492e-4792-aebf-92a1af154138 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.746999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc392c5c-a0c6-46b0-8cd9-59f62523ecc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.747622 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc392c5c-a0c6-46b0-8cd9-59f62523ecc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.747901 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc392c5c-a0c6-46b0-8cd9-59f62523ecc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.748173 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc392c5c-a0c6-46b0-8cd9-59f62523ecc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.748627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16b84c4-f137-4d7d-b5bd-b300f15de1bd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.748812 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc392c5c-a0c6-46b0-8cd9-59f62523ecc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.750502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc392c5c-a0c6-46b0-8cd9-59f62523ecc3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.754067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8dcd2-279c-47f9-bf4e-d8c3fb015175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.754761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8dcd2-279c-47f9-bf4e-d8c3fb015175 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.755049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8dcd2-279c-47f9-bf4e-d8c3fb015175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.755380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8dcd2-279c-47f9-bf4e-d8c3fb015175 admin admin] PATH_INFO: `/v3/projects/7e39c6ca75a3466d93195fe245509bc9/users/56535a702f4344aa8a58f11406acf4c9/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.761950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 802/1605] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:22.762627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc392c5c-a0c6-46b0-8cd9-59f62523ecc3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.771039 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8dcd2-279c-47f9-bf4e-d8c3fb015175 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e39c6ca75a3466d93195fe245509bc9, user_id=56535a702f4344aa8a58f11406acf4c9, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.772251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc392c5c-a0c6-46b0-8cd9-59f62523ecc3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.773726 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8dcd2-279c-47f9-bf4e-d8c3fb015175 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.776905 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efe9d3-2991-4ab9-9b24-62f7175cf262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.777494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efe9d3-2991-4ab9-9b24-62f7175cf262 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.777820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efe9d3-2991-4ab9-9b24-62f7175cf262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.778162 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4efe9d3-2991-4ab9-9b24-62f7175cf262 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.779133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4efe9d3-2991-4ab9-9b24-62f7175cf262 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.781119 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4efe9d3-2991-4ab9-9b24-62f7175cf262 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.787250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 804/1606] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:22 2022] PUT /identity/v3/projects/7e39c6ca75a3466d93195fe245509bc9/users/56535a702f4344aa8a58f11406acf4c9/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:22.788049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8dcd2-279c-47f9-bf4e-d8c3fb015175 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.797842 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 803/1607] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:22.797842 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efe9d3-2991-4ab9-9b24-62f7175cf262 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.808349 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4efe9d3-2991-4ab9-9b24-62f7175cf262 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.850364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8dcd2-279c-47f9-bf4e-d8c3fb015175 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.855152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7449b0-405b-412d-a3c5-28a100e9b3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.855818 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7449b0-405b-412d-a3c5-28a100e9b3fd admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.856115 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7449b0-405b-412d-a3c5-28a100e9b3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.856376 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7449b0-405b-412d-a3c5-28a100e9b3fd admin admin] PATH_INFO: `/v3/projects/e1d7efa8baf2432886619cfe25aa706d/users/bb79ba1175694ad5928444f66064a1e2/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.867784 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a6160a-4b06-4cfd-a0b6-2aac74d4ecd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:22.868438 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6160a-4b06-4cfd-a0b6-2aac74d4ecd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.868715 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6160a-4b06-4cfd-a0b6-2aac74d4ecd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.869009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a6160a-4b06-4cfd-a0b6-2aac74d4ecd3 admin admin] PATH_INFO: `/v3/projects/77bcbb8b4a9b44299ca3ac710c1fdee7/users/172faa476a6f4d10b11ad91f828a2ae5/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.873573 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7449b0-405b-412d-a3c5-28a100e9b3fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1d7efa8baf2432886619cfe25aa706d, user_id=bb79ba1175694ad5928444f66064a1e2, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.876883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7449b0-405b-412d-a3c5-28a100e9b3fd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.884615 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6160a-4b06-4cfd-a0b6-2aac74d4ecd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77bcbb8b4a9b44299ca3ac710c1fdee7, user_id=172faa476a6f4d10b11ad91f828a2ae5, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:22.887263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a6160a-4b06-4cfd-a0b6-2aac74d4ecd3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:22.890618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 805/1608] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:22 2022] PUT /identity/v3/projects/e1d7efa8baf2432886619cfe25aa706d/users/bb79ba1175694ad5928444f66064a1e2/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:22.899096 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7449b0-405b-412d-a3c5-28a100e9b3fd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:22.909837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 804/1609] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:22 2022] PUT /identity/v3/projects/77bcbb8b4a9b44299ca3ac710c1fdee7/users/172faa476a6f4d10b11ad91f828a2ae5/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:22.909837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe6cf8-c068-485c-be61-bd76327f45bb None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:22.909837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe6cf8-c068-485c-be61-bd76327f45bb None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:22.909837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fe6cf8-c068-485c-be61-bd76327f45bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:22.935245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b8fe6cf8-c068-485c-be61-bd76327f45bb None None] MFA Rules not processed for user `be45cc3aa12e44a4b6d49976b6eff0b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:22.936587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b8fe6cf8-c068-485c-be61-bd76327f45bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:22.963963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:22.977702 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7449b0-405b-412d-a3c5-28a100e9b3fd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:22.982783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:23.008483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 805/1610] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:23.009453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe6cf8-c068-485c-be61-bd76327f45bb None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:23.044220 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaea2c-9b45-420d-a4a7-f132b3192475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:23.044796 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaea2c-9b45-420d-a4a7-f132b3192475 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.045070 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaea2c-9b45-420d-a4a7-f132b3192475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.045335 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27aaea2c-9b45-420d-a4a7-f132b3192475 admin admin] PATH_INFO: `/v3/projects/6a03c5044d764f21afc9b56b56e9a5d2/users/4be80290e6ab4299ba70931bb1cd8f03/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.047837 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aaea2c-9b45-420d-a4a7-f132b3192475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a03c5044d764f21afc9b56b56e9a5d2, user_id=4be80290e6ab4299ba70931bb1cd8f03, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:23.050554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27aaea2c-9b45-420d-a4a7-f132b3192475 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:23.060764 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 806/1611] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:22 2022] PUT /identity/v3/projects/6a03c5044d764f21afc9b56b56e9a5d2/users/4be80290e6ab4299ba70931bb1cd8f03/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:23.061548 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaea2c-9b45-420d-a4a7-f132b3192475 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:23.078509 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fe6cf8-c068-485c-be61-bd76327f45bb None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:23.132569 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27aaea2c-9b45-420d-a4a7-f132b3192475 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:23.134983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25098e43-a60b-48bb-a6d7-73054c403037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:23.135590 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25098e43-a60b-48bb-a6d7-73054c403037 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.135867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25098e43-a60b-48bb-a6d7-73054c403037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.136137 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25098e43-a60b-48bb-a6d7-73054c403037 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.136749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25098e43-a60b-48bb-a6d7-73054c403037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:23.137479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478400ae-f2ca-4eda-85eb-7f7c78b5b0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:23.138157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478400ae-f2ca-4eda-85eb-7f7c78b5b0ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.138529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478400ae-f2ca-4eda-85eb-7f7c78b5b0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.138700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25098e43-a60b-48bb-a6d7-73054c403037 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:23.138821 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478400ae-f2ca-4eda-85eb-7f7c78b5b0ec admin admin] PATH_INFO: `/v3/projects/b8c52f09126b464fae2407602e740b08/users/5c0d72599f8b4f0ab924c11f50b8c3f6/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.153483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478400ae-f2ca-4eda-85eb-7f7c78b5b0ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8c52f09126b464fae2407602e740b08, user_id=5c0d72599f8b4f0ab924c11f50b8c3f6, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:23.156958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478400ae-f2ca-4eda-85eb-7f7c78b5b0ec admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:23.161846 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 806/1612] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:26:23 2022] POST /identity/v3/users => generated 297 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:23.164773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc849764-5775-40eb-814d-3ff3a09cf0b8 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.165034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc849764-5775-40eb-814d-3ff3a09cf0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.165297 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc849764-5775-40eb-814d-3ff3a09cf0b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.170689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 807/1613] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:23 2022] PUT /identity/v3/projects/b8c52f09126b464fae2407602e740b08/users/5c0d72599f8b4f0ab924c11f50b8c3f6/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:23.172232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478400ae-f2ca-4eda-85eb-7f7c78b5b0ec admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:23.192227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-dc849764-5775-40eb-814d-3ff3a09cf0b8 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:23.193462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-dc849764-5775-40eb-814d-3ff3a09cf0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:23.238279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478400ae-f2ca-4eda-85eb-7f7c78b5b0ec admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:23.244063 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35828d85-9ec1-4420-9278-50dacc4b003b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:23.244721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35828d85-9ec1-4420-9278-50dacc4b003b admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.244995 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35828d85-9ec1-4420-9278-50dacc4b003b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.245266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35828d85-9ec1-4420-9278-50dacc4b003b admin admin] PATH_INFO: `/v3/projects/fd3f4fde69fc4507837f9feb27ceed01/users/b7a4e434c3c74b239c50da6b5d500c23/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.253406 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 807/1614] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:23.254679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee92316-202e-4c28-b1b8-b863f03d941b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.254964 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee92316-202e-4c28-b1b8-b863f03d941b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.255235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee92316-202e-4c28-b1b8-b863f03d941b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.264725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35828d85-9ec1-4420-9278-50dacc4b003b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd3f4fde69fc4507837f9feb27ceed01, user_id=b7a4e434c3c74b239c50da6b5d500c23, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:23.267493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35828d85-9ec1-4420-9278-50dacc4b003b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:23.278584 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:23.284125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0ee92316-202e-4c28-b1b8-b863f03d941b None None] MFA Rules not processed for user `bb79ba1175694ad5928444f66064a1e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:23.284522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 808/1615] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:23 2022] PUT /identity/v3/projects/fd3f4fde69fc4507837f9feb27ceed01/users/b7a4e434c3c74b239c50da6b5d500c23/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:23.285377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0ee92316-202e-4c28-b1b8-b863f03d941b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:23.286303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce5ae3-16d0-4edc-94c3-24ae8de29e79 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.286646 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce5ae3-16d0-4edc-94c3-24ae8de29e79 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.286986 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce5ae3-16d0-4edc-94c3-24ae8de29e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.297167 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:23.321015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-07ce5ae3-16d0-4edc-94c3-24ae8de29e79 None None] MFA Rules not processed for user `172faa476a6f4d10b11ad91f828a2ae5`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:23.322284 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-07ce5ae3-16d0-4edc-94c3-24ae8de29e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:23.366476 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 808/1616] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:23 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:23.367474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee92316-202e-4c28-b1b8-b863f03d941b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:23.390763 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 809/1617] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:23.391991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ee96e-c25d-41c5-86a4-aa11c0440522 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.392278 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ee96e-c25d-41c5-86a4-aa11c0440522 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.392547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ee96e-c25d-41c5-86a4-aa11c0440522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.424134 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c17ee96e-c25d-41c5-86a4-aa11c0440522 None None] MFA Rules not processed for user `4be80290e6ab4299ba70931bb1cd8f03`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:23.425417 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c17ee96e-c25d-41c5-86a4-aa11c0440522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:23.444632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee92316-202e-4c28-b1b8-b863f03d941b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:23.449439 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e4eee-9ee9-4dd4-bdb8-fc0c70dbbf81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:23.450053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e4eee-9ee9-4dd4-bdb8-fc0c70dbbf81 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.450366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e4eee-9ee9-4dd4-bdb8-fc0c70dbbf81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.450640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e4eee-9ee9-4dd4-bdb8-fc0c70dbbf81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.451268 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e4eee-9ee9-4dd4-bdb8-fc0c70dbbf81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:23.453688 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e4eee-9ee9-4dd4-bdb8-fc0c70dbbf81 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:23.482549 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 809/1618] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:26:23 2022] POST /identity/v3/projects => generated 377 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:23.483404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e4eee-9ee9-4dd4-bdb8-fc0c70dbbf81 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:23.500632 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 810/1619] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:23.502062 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0f60d-8ea5-4c39-9bd1-a997a9f39b9b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.502386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0f60d-8ea5-4c39-9bd1-a997a9f39b9b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.502665 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df0f60d-8ea5-4c39-9bd1-a997a9f39b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.531478 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3df0f60d-8ea5-4c39-9bd1-a997a9f39b9b None None] MFA Rules not processed for user `5c0d72599f8b4f0ab924c11f50b8c3f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:23.532957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3df0f60d-8ea5-4c39-9bd1-a997a9f39b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:23.556663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e4eee-9ee9-4dd4-bdb8-fc0c70dbbf81 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:23.561221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5fb29-d1d5-474e-9a3a-4a404a383e73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:23.561826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5fb29-d1d5-474e-9a3a-4a404a383e73 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.562128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5fb29-d1d5-474e-9a3a-4a404a383e73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.563143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5fb29-d1d5-474e-9a3a-4a404a383e73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.563716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5fb29-d1d5-474e-9a3a-4a404a383e73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:23.565614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5fb29-d1d5-474e-9a3a-4a404a383e73 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:23.575937 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 810/1620] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:23 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:23.576835 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5fb29-d1d5-474e-9a3a-4a404a383e73 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:23.606496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 811/1621] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:23.608232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca26b3-5d28-4ff3-90f7-c551b4105dcd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.608532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca26b3-5d28-4ff3-90f7-c551b4105dcd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.608805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca26b3-5d28-4ff3-90f7-c551b4105dcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.616674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:23.619900 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:23.634733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-edca26b3-5d28-4ff3-90f7-c551b4105dcd None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:23.635953 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-edca26b3-5d28-4ff3-90f7-c551b4105dcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:23.642050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5fb29-d1d5-474e-9a3a-4a404a383e73 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:23.644889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 812/1622] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:23.645708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca26b3-5d28-4ff3-90f7-c551b4105dcd None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:23.649288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff25b8-c7c6-4b4d-8fcc-f39f217577aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:23.650293 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff25b8-c7c6-4b4d-8fcc-f39f217577aa service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.650793 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff25b8-c7c6-4b4d-8fcc-f39f217577aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.651372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff25b8-c7c6-4b4d-8fcc-f39f217577aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.662727 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff25b8-c7c6-4b4d-8fcc-f39f217577aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:23.664242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff25b8-c7c6-4b4d-8fcc-f39f217577aa service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:23.676614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 811/1623] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:23.680653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae451719-0707-4a09-8b8d-ca5a26dc0a25 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.680936 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae451719-0707-4a09-8b8d-ca5a26dc0a25 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.681210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae451719-0707-4a09-8b8d-ca5a26dc0a25 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.682698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 812/1624] 10.222.0.76 () {54 vars in 962 bytes} [Wed Aug 24 15:26:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:23.683469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae451719-0707-4a09-8b8d-ca5a26dc0a25 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:23.706381 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae451719-0707-4a09-8b8d-ca5a26dc0a25 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:23.711537 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edca26b3-5d28-4ff3-90f7-c551b4105dcd None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:23.716944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9920a3d-d575-479e-a077-e885c637e2e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:23.717783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9920a3d-d575-479e-a077-e885c637e2e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.718112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9920a3d-d575-479e-a077-e885c637e2e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.718511 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9920a3d-d575-479e-a077-e885c637e2e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.728242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a747fd9-47c3-4a15-ae00-6535f6dff9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:23.728869 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a747fd9-47c3-4a15-ae00-6535f6dff9e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.729153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a747fd9-47c3-4a15-ae00-6535f6dff9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.729426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a747fd9-47c3-4a15-ae00-6535f6dff9e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.730061 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a747fd9-47c3-4a15-ae00-6535f6dff9e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:23.732697 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a747fd9-47c3-4a15-ae00-6535f6dff9e4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:23.758801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 813/1625] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:26:23 2022] POST /identity/v3/users => generated 295 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:23.759994 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a747fd9-47c3-4a15-ae00-6535f6dff9e4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:23.775041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a747fd9-47c3-4a15-ae00-6535f6dff9e4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:23.779409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f45aec-aac1-4320-8289-44952fe99510 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:23.780213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f45aec-aac1-4320-8289-44952fe99510 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.780598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f45aec-aac1-4320-8289-44952fe99510 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.781005 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f45aec-aac1-4320-8289-44952fe99510 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.795274 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9920a3d-d575-479e-a077-e885c637e2e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:23.796967 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9920a3d-d575-479e-a077-e885c637e2e6 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:23.807891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 813/1626] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:23 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:23.808883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9920a3d-d575-479e-a077-e885c637e2e6 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:23.822708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9920a3d-d575-479e-a077-e885c637e2e6 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:23.828810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec56df8-a1b3-4cec-9a32-81098ef9515c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:23.829688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec56df8-a1b3-4cec-9a32-81098ef9515c admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.830126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec56df8-a1b3-4cec-9a32-81098ef9515c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.830577 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec56df8-a1b3-4cec-9a32-81098ef9515c admin admin] PATH_INFO: `/v3/projects/3fffff519d354bd6a8ee7250b8cc3b87/users/c86122cbced04d2da61e3067964a211f/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.858295 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec56df8-a1b3-4cec-9a32-81098ef9515c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fffff519d354bd6a8ee7250b8cc3b87, user_id=c86122cbced04d2da61e3067964a211f, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:23.861401 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec56df8-a1b3-4cec-9a32-81098ef9515c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:23.865952 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f45aec-aac1-4320-8289-44952fe99510 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:23.868656 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f45aec-aac1-4320-8289-44952fe99510 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:23.880028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 814/1627] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:23 2022] PUT /identity/v3/projects/3fffff519d354bd6a8ee7250b8cc3b87/users/c86122cbced04d2da61e3067964a211f/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:23.881531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca7850-a17e-4076-ba37-82a2cd58b580 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.881983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca7850-a17e-4076-ba37-82a2cd58b580 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.882492 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca7850-a17e-4076-ba37-82a2cd58b580 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.884317 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 815/1628] 10.222.0.76 () {54 vars in 962 bytes} [Wed Aug 24 15:26:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:23.885252 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca7850-a17e-4076-ba37-82a2cd58b580 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:23.899074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 814/1629] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:23.900798 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a465880-b65b-4c1b-8103-f4f8434656f9 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.902466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a465880-b65b-4c1b-8103-f4f8434656f9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.909113 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a465880-b65b-4c1b-8103-f4f8434656f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:23.983945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca7850-a17e-4076-ba37-82a2cd58b580 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:23.992048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8a465880-b65b-4c1b-8103-f4f8434656f9 None None] MFA Rules not processed for user `8a5fd31a2db944f596912e4f0aa26835`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:23.992048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8a465880-b65b-4c1b-8103-f4f8434656f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:23.993850 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003ed08-31ea-4dbf-aa6f-b29e12c3442c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:23.994513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003ed08-31ea-4dbf-aa6f-b29e12c3442c service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:23.994785 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003ed08-31ea-4dbf-aa6f-b29e12c3442c service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:23.995059 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5003ed08-31ea-4dbf-aa6f-b29e12c3442c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:24.066058 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:24.109796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 815/1630] 10.222.0.76 () {58 vars in 1059 bytes} [Wed Aug 24 15:26:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:24.110151 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003ed08-31ea-4dbf-aa6f-b29e12c3442c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:24.111703 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5003ed08-31ea-4dbf-aa6f-b29e12c3442c service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:24.131457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a465880-b65b-4c1b-8103-f4f8434656f9 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:24.154763 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a465880-b65b-4c1b-8103-f4f8434656f9 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:24.161509 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 816/1631] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:24.163558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003ed08-31ea-4dbf-aa6f-b29e12c3442c service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:24.166503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aad339-8c9b-48b1-93e1-c661404db027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:24.168961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aad339-8c9b-48b1-93e1-c661404db027 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:24.169235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aad339-8c9b-48b1-93e1-c661404db027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:24.169512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aad339-8c9b-48b1-93e1-c661404db027 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:24.172021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aad339-8c9b-48b1-93e1-c661404db027 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:24.177051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aad339-8c9b-48b1-93e1-c661404db027 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:24.189718 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5003ed08-31ea-4dbf-aa6f-b29e12c3442c service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:24.202635 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6aff97-d9fc-4930-a0c5-5fafe64fc0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:24.204178 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 816/1632] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:24.204479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6aff97-d9fc-4930-a0c5-5fafe64fc0bd admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:24.205401 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6aff97-d9fc-4930-a0c5-5fafe64fc0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:24.205584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092ee4c-54ac-4f07-b2c3-80a4564ba3cc None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:24.205971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092ee4c-54ac-4f07-b2c3-80a4564ba3cc None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:24.206588 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6aff97-d9fc-4930-a0c5-5fafe64fc0bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:24.208211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092ee4c-54ac-4f07-b2c3-80a4564ba3cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:24.209032 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6aff97-d9fc-4930-a0c5-5fafe64fc0bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:24.211901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6aff97-d9fc-4930-a0c5-5fafe64fc0bd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:24.243119 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 817/1633] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:24.245689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6aff97-d9fc-4930-a0c5-5fafe64fc0bd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:24.254538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5092ee4c-54ac-4f07-b2c3-80a4564ba3cc None None] MFA Rules not processed for user `8a5fd31a2db944f596912e4f0aa26835`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:24.255957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5092ee4c-54ac-4f07-b2c3-80a4564ba3cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:24.273539 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6aff97-d9fc-4930-a0c5-5fafe64fc0bd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:24.281035 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 817/1634] 10.222.0.76 () {58 vars in 1059 bytes} [Wed Aug 24 15:26:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:24.282639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092ee4c-54ac-4f07-b2c3-80a4564ba3cc None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:24.295873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092ee4c-54ac-4f07-b2c3-80a4564ba3cc None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:24.298919 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff855f-d122-4160-9b77-93d50eee7178 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:24.299550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff855f-d122-4160-9b77-93d50eee7178 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:24.299862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff855f-d122-4160-9b77-93d50eee7178 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:24.300090 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aff855f-d122-4160-9b77-93d50eee7178 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:24.303335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222299c-251c-4c04-b6e5-551493aee35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:24.304292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222299c-251c-4c04-b6e5-551493aee35f admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:24.304697 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222299c-251c-4c04-b6e5-551493aee35f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:24.305069 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222299c-251c-4c04-b6e5-551493aee35f admin admin] PATH_INFO: `/v3/projects/d62f5051ce9d4486a0fdb807a9810da1/users/888d603152ea4ed29c5558b6763beb9c/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:24.343940 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:24.353133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222299c-251c-4c04-b6e5-551493aee35f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62f5051ce9d4486a0fdb807a9810da1, user_id=888d603152ea4ed29c5558b6763beb9c, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:24.360613 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1222299c-251c-4c04-b6e5-551493aee35f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:24.389900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 818/1635] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:24 2022] PUT /identity/v3/projects/d62f5051ce9d4486a0fdb807a9810da1/users/888d603152ea4ed29c5558b6763beb9c/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:24.391517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222299c-251c-4c04-b6e5-551493aee35f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:24.491842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff855f-d122-4160-9b77-93d50eee7178 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:24.497143 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aff855f-d122-4160-9b77-93d50eee7178 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:24.512153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222299c-251c-4c04-b6e5-551493aee35f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:24.519693 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d998b8-ff04-499d-befe-3ade6e6cbc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:24.520695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d998b8-ff04-499d-befe-3ade6e6cbc85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:24.521352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d998b8-ff04-499d-befe-3ade6e6cbc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:24.521958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d998b8-ff04-499d-befe-3ade6e6cbc85 admin admin] PATH_INFO: `/v3/projects/3fffff519d354bd6a8ee7250b8cc3b87/users/c86122cbced04d2da61e3067964a211f/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:24.534162 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 818/1636] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:24.535331 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff855f-d122-4160-9b77-93d50eee7178 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:24.547658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d998b8-ff04-499d-befe-3ade6e6cbc85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fffff519d354bd6a8ee7250b8cc3b87, user_id=c86122cbced04d2da61e3067964a211f, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:24.556403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d998b8-ff04-499d-befe-3ade6e6cbc85 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:24.566529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aff855f-d122-4160-9b77-93d50eee7178 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:24.593161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 819/1637] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:24 2022] PUT /identity/v3/projects/3fffff519d354bd6a8ee7250b8cc3b87/users/c86122cbced04d2da61e3067964a211f/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:24.593161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d998b8-ff04-499d-befe-3ade6e6cbc85 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:24.765128 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac3416-daf3-47ff-9ea5-b9b01e73e219 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:24.767000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac3416-daf3-47ff-9ea5-b9b01e73e219 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:24.767776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac3416-daf3-47ff-9ea5-b9b01e73e219 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:24.768358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac3416-daf3-47ff-9ea5-b9b01e73e219 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:24.772161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d998b8-ff04-499d-befe-3ade6e6cbc85 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:24.787516 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d6f43-e1de-4409-8b7c-356515895bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:24.788290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d6f43-e1de-4409-8b7c-356515895bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:24.788769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d6f43-e1de-4409-8b7c-356515895bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:24.789235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d6f43-e1de-4409-8b7c-356515895bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:24.790210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d6f43-e1de-4409-8b7c-356515895bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:24.794845 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d6f43-e1de-4409-8b7c-356515895bb1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:24.827458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 820/1638] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:24.827458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e851a-1139-4d0e-bfe8-023f2b7a8535 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:24.827458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e851a-1139-4d0e-bfe8-023f2b7a8535 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:24.827458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373e851a-1139-4d0e-bfe8-023f2b7a8535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:24.848938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ac3416-daf3-47ff-9ea5-b9b01e73e219 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:24.851179 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ac3416-daf3-47ff-9ea5-b9b01e73e219 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:24.866932 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 819/1639] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:24.867920 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac3416-daf3-47ff-9ea5-b9b01e73e219 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:24.874202 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:24.884540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac3416-daf3-47ff-9ea5-b9b01e73e219 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:24.889457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226c2d2b-ea82-4a01-ae1e-53828f9c7864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:24.890112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c2d2b-ea82-4a01-ae1e-53828f9c7864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:24.890534 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c2d2b-ea82-4a01-ae1e-53828f9c7864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:24.916794 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226c2d2b-ea82-4a01-ae1e-53828f9c7864 admin admin] PATH_INFO: `/v3/users/5579431178d0499188de35c56b925a63` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:24.920949 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c2d2b-ea82-4a01-ae1e-53828f9c7864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5579431178d0499188de35c56b925a63)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:24.927081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-373e851a-1139-4d0e-bfe8-023f2b7a8535 None None] MFA Rules not processed for user `c86122cbced04d2da61e3067964a211f`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:24.928982 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226c2d2b-ea82-4a01-ae1e-53828f9c7864 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:24.933635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-373e851a-1139-4d0e-bfe8-023f2b7a8535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:25.077593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-226c2d2b-ea82-4a01-ae1e-53828f9c7864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5579431178d0499188de35c56b925a63'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:25.078916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-226c2d2b-ea82-4a01-ae1e-53828f9c7864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5579431178d0499188de35c56b925a63'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:25.127193 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-226c2d2b-ea82-4a01-ae1e-53828f9c7864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5579431178d0499188de35c56b925a63'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:25.144435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-226c2d2b-ea82-4a01-ae1e-53828f9c7864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5579431178d0499188de35c56b925a63'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:25.145268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-226c2d2b-ea82-4a01-ae1e-53828f9c7864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5579431178d0499188de35c56b925a63'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:25.155601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 821/1640] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:25.156894 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e851a-1139-4d0e-bfe8-023f2b7a8535 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:25.173362 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 820/1641] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:26:24 2022] DELETE /identity/v3/users/5579431178d0499188de35c56b925a63 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:25.173966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:25.181537 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33841cd3-088f-4292-ae07-a54507cd23b8 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:25.181830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33841cd3-088f-4292-ae07-a54507cd23b8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:25.182293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33841cd3-088f-4292-ae07-a54507cd23b8 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:25.183998 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 821/1642] 10.222.0.76 () {54 vars in 912 bytes} [Wed Aug 24 15:26:25 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:25.189223 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33841cd3-088f-4292-ae07-a54507cd23b8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:25.193677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-33841cd3-088f-4292-ae07-a54507cd23b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:25.193959 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373e851a-1139-4d0e-bfe8-023f2b7a8535 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:25.337644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ae57f1-ea08-4367-a05f-44d715f95bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:25.338330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae57f1-ea08-4367-a05f-44d715f95bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:25.339202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae57f1-ea08-4367-a05f-44d715f95bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:25.339486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ae57f1-ea08-4367-a05f-44d715f95bd5 admin admin] PATH_INFO: `/v3/projects/d62f5051ce9d4486a0fdb807a9810da1/users/888d603152ea4ed29c5558b6763beb9c/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:25.342183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33841cd3-088f-4292-ae07-a54507cd23b8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:25.344274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae57f1-ea08-4367-a05f-44d715f95bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62f5051ce9d4486a0fdb807a9810da1, user_id=888d603152ea4ed29c5558b6763beb9c, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:25.350446 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b48a5f-3e7e-4f4f-ae7f-f2ed0aa8da75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:25.351085 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b48a5f-3e7e-4f4f-ae7f-f2ed0aa8da75 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:25.351365 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b48a5f-3e7e-4f4f-ae7f-f2ed0aa8da75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:25.351635 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b48a5f-3e7e-4f4f-ae7f-f2ed0aa8da75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:25.353875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ae57f1-ea08-4367-a05f-44d715f95bd5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:25.360557 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-21b48a5f-3e7e-4f4f-ae7f-f2ed0aa8da75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:25.395374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 822/1643] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:25 2022] PUT /identity/v3/projects/d62f5051ce9d4486a0fdb807a9810da1/users/888d603152ea4ed29c5558b6763beb9c/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:25.397229 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8562e-c993-4338-b13f-fdcf49d2df75 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:25.397528 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8562e-c993-4338-b13f-fdcf49d2df75 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:25.397801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac8562e-c993-4338-b13f-fdcf49d2df75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:25.450949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3ac8562e-c993-4338-b13f-fdcf49d2df75 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:25.454681 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3ac8562e-c993-4338-b13f-fdcf49d2df75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:25.488469 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b48a5f-3e7e-4f4f-ae7f-f2ed0aa8da75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:25.492168 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b48a5f-3e7e-4f4f-ae7f-f2ed0aa8da75 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:25.521198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 822/1644] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:25 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:25.524690 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a270589-342b-4b2c-903c-90e2465536c6 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:25.525184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a270589-342b-4b2c-903c-90e2465536c6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:25.525533 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a270589-342b-4b2c-903c-90e2465536c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:25.548196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 823/1645] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:25.549707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d50ea2-e2ed-41f1-9df8-c6b9ccdd10ca None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:25.550001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d50ea2-e2ed-41f1-9df8-c6b9ccdd10ca None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:25.550425 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d50ea2-e2ed-41f1-9df8-c6b9ccdd10ca None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:25.552668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 824/1646] 10.222.0.76 () {54 vars in 912 bytes} [Wed Aug 24 15:26:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:25.554665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b936d-c0d2-4e72-854e-47b5c259f483 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:25.554665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b936d-c0d2-4e72-854e-47b5c259f483 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:25.554969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1b936d-c0d2-4e72-854e-47b5c259f483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:25.579597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1a270589-342b-4b2c-903c-90e2465536c6 None None] MFA Rules not processed for user `8a5fd31a2db944f596912e4f0aa26835`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:25.584048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1a270589-342b-4b2c-903c-90e2465536c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:25.634798 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7f1b936d-c0d2-4e72-854e-47b5c259f483 None None] MFA Rules not processed for user `888d603152ea4ed29c5558b6763beb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:25.646866 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7f1b936d-c0d2-4e72-854e-47b5c259f483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:25.661862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:25.809793 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 825/1647] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:25 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:25.811261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7d55f-7119-411b-950c-24450409ef42 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:25.811553 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7d55f-7119-411b-950c-24450409ef42 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:25.811826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7d55f-7119-411b-950c-24450409ef42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:25.834622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 823/1648] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 15:26:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:25.834966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3af7d55f-7119-411b-950c-24450409ef42 None None] MFA Rules not processed for user `8a5fd31a2db944f596912e4f0aa26835`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:25.835991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a270589-342b-4b2c-903c-90e2465536c6 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:25.836274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3af7d55f-7119-411b-950c-24450409ef42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:25.839080 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1a270589-342b-4b2c-903c-90e2465536c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:25.855859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 826/1649] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 15:26:25 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:25.857352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7d55f-7119-411b-950c-24450409ef42 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:25.894904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:25.953406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a270589-342b-4b2c-903c-90e2465536c6 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:25.958200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd43b34-6740-450b-94d2-e8491a93fd4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:25.958890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd43b34-6740-450b-94d2-e8491a93fd4f service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:25.959236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd43b34-6740-450b-94d2-e8491a93fd4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:25.959715 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd43b34-6740-450b-94d2-e8491a93fd4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:25.973037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd43b34-6740-450b-94d2-e8491a93fd4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:25.974621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd43b34-6740-450b-94d2-e8491a93fd4f service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:25.990912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7d55f-7119-411b-950c-24450409ef42 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:25.991618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 824/1650] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:25.992657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd43b34-6740-450b-94d2-e8491a93fd4f service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:25.994685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0dd43b34-6740-450b-94d2-e8491a93fd4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:26.001470 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb021546-5ffc-4f1b-8183-e8728c8e38e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:26.002128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb021546-5ffc-4f1b-8183-e8728c8e38e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:26.002441 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb021546-5ffc-4f1b-8183-e8728c8e38e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:26.002717 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb021546-5ffc-4f1b-8183-e8728c8e38e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:26.008640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-eb021546-5ffc-4f1b-8183-e8728c8e38e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:26.013243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd43b34-6740-450b-94d2-e8491a93fd4f service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:26.017683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b8147-b8fa-4207-9f51-fedc8200f196 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:26.018493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b8147-b8fa-4207-9f51-fedc8200f196 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:26.018853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b8147-b8fa-4207-9f51-fedc8200f196 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:26.019935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0b8147-b8fa-4207-9f51-fedc8200f196 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:26.022487 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bc0b8147-b8fa-4207-9f51-fedc8200f196 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:26.130449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb021546-5ffc-4f1b-8183-e8728c8e38e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:26.132010 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb021546-5ffc-4f1b-8183-e8728c8e38e4 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:26.140141 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b8147-b8fa-4207-9f51-fedc8200f196 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:26.142407 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0b8147-b8fa-4207-9f51-fedc8200f196 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:26.157098 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 827/1651] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:25 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:26.179570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 825/1652] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:26.445594 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb021546-5ffc-4f1b-8183-e8728c8e38e4 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:26.455186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb021546-5ffc-4f1b-8183-e8728c8e38e4 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:26.464212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8eaecb-3da9-4faa-b211-60dd1fd3d4fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:26.465028 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eaecb-3da9-4faa-b211-60dd1fd3d4fd service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:26.465356 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eaecb-3da9-4faa-b211-60dd1fd3d4fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:26.465583 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8eaecb-3da9-4faa-b211-60dd1fd3d4fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:26.490570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8eaecb-3da9-4faa-b211-60dd1fd3d4fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:26.492043 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8eaecb-3da9-4faa-b211-60dd1fd3d4fd service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:26.521531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 828/1653] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:26.661516 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b8147-b8fa-4207-9f51-fedc8200f196 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:26.663484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bc0b8147-b8fa-4207-9f51-fedc8200f196 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:26.693631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0b8147-b8fa-4207-9f51-fedc8200f196 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:26.708065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:26.709203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:26.709599 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:26.710532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7 admin admin] PATH_INFO: `/v3/projects/c237131ebfac4007b296b5dd564e89e5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:26.715202 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c237131ebfac4007b296b5dd564e89e5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:26.716744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: WARNING py.warnings [None req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:26:26.716744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: warnings.warn(msg) Aug 24 15:26:26.716744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: Aug 24 15:26:26.717993 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:26.889495 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c237131ebfac4007b296b5dd564e89e5', 'request_id': 'req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:26.901206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:26.923209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c237131ebfac4007b296b5dd564e89e5', 'request_id': 'req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:26.955963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c237131ebfac4007b296b5dd564e89e5', 'request_id': 'req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:27.021362 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7 admin admin] The token cache is being invalidate because project c237131ebfac4007b296b5dd564e89e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:26:27.021663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:27.024350 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:27.031207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 826/1654] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:26:26 2022] DELETE /identity/v3/projects/c237131ebfac4007b296b5dd564e89e5 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:27.035354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8eaecb-3da9-4faa-b211-60dd1fd3d4fd service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:27.053913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:27.055603 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8eaecb-3da9-4faa-b211-60dd1fd3d4fd service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:27.058317 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:27.279616 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec9ef71-5a40-4243-8657-ebde346b5d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:27.281028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab78e92-9fc1-4355-a564-4ef34f2c4ed7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:27.285725 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9ef71-5a40-4243-8657-ebde346b5d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:27.286058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9ef71-5a40-4243-8657-ebde346b5d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:27.286363 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec9ef71-5a40-4243-8657-ebde346b5d0e admin admin] PATH_INFO: `/v3/users/4be80290e6ab4299ba70931bb1cd8f03` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:27.290540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9ef71-5a40-4243-8657-ebde346b5d0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4be80290e6ab4299ba70931bb1cd8f03)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:27.293853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: WARNING py.warnings [None req-4ec9ef71-5a40-4243-8657-ebde346b5d0e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 15:26:27.293853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: warnings.warn(msg) Aug 24 15:26:27.293853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: Aug 24 15:26:27.297462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466083d8-04b7-4332-a3dc-fdd150685170 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:27.298119 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466083d8-04b7-4332-a3dc-fdd150685170 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:27.298425 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466083d8-04b7-4332-a3dc-fdd150685170 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:27.298699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466083d8-04b7-4332-a3dc-fdd150685170 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:27.302569 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-466083d8-04b7-4332-a3dc-fdd150685170 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:27.309005 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec9ef71-5a40-4243-8657-ebde346b5d0e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:27.452190 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466083d8-04b7-4332-a3dc-fdd150685170 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:27.454838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466083d8-04b7-4332-a3dc-fdd150685170 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:27.479311 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 827/1655] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:27.637477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4ec9ef71-5a40-4243-8657-ebde346b5d0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4be80290e6ab4299ba70931bb1cd8f03'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:27.637477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4ec9ef71-5a40-4243-8657-ebde346b5d0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4be80290e6ab4299ba70931bb1cd8f03'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:27.646367 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:27.665780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4ec9ef71-5a40-4243-8657-ebde346b5d0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4be80290e6ab4299ba70931bb1cd8f03'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:27.666546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4ec9ef71-5a40-4243-8657-ebde346b5d0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4be80290e6ab4299ba70931bb1cd8f03'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:27.704991 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4ec9ef71-5a40-4243-8657-ebde346b5d0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4be80290e6ab4299ba70931bb1cd8f03'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:27.730485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 829/1656] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:26:27 2022] DELETE /identity/v3/users/4be80290e6ab4299ba70931bb1cd8f03 => generated 0 bytes in 696 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:28.795453 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466083d8-04b7-4332-a3dc-fdd150685170 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:28.798692 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-466083d8-04b7-4332-a3dc-fdd150685170 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:28.846385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466083d8-04b7-4332-a3dc-fdd150685170 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:29.081906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b58d89-4acc-4703-bf0c-01cdcb7427bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:29.082713 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b58d89-4acc-4703-bf0c-01cdcb7427bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:29.082939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b58d89-4acc-4703-bf0c-01cdcb7427bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:29.083215 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b58d89-4acc-4703-bf0c-01cdcb7427bd admin admin] PATH_INFO: `/v3/projects/6a03c5044d764f21afc9b56b56e9a5d2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:29.092482 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b58d89-4acc-4703-bf0c-01cdcb7427bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a03c5044d764f21afc9b56b56e9a5d2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:29.113158 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b58d89-4acc-4703-bf0c-01cdcb7427bd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:29.159632 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:29.214540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f9b58d89-4acc-4703-bf0c-01cdcb7427bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a03c5044d764f21afc9b56b56e9a5d2', 'request_id': 'req-f9b58d89-4acc-4703-bf0c-01cdcb7427bd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:29.222307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f9b58d89-4acc-4703-bf0c-01cdcb7427bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a03c5044d764f21afc9b56b56e9a5d2', 'request_id': 'req-f9b58d89-4acc-4703-bf0c-01cdcb7427bd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:29.240800 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f9b58d89-4acc-4703-bf0c-01cdcb7427bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a03c5044d764f21afc9b56b56e9a5d2', 'request_id': 'req-f9b58d89-4acc-4703-bf0c-01cdcb7427bd'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:29.256759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f9b58d89-4acc-4703-bf0c-01cdcb7427bd admin admin] The token cache is being invalidate because project 6a03c5044d764f21afc9b56b56e9a5d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:26:29.257693 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f9b58d89-4acc-4703-bf0c-01cdcb7427bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:29.258426 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f9b58d89-4acc-4703-bf0c-01cdcb7427bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:29.263696 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 828/1657] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:26:28 2022] DELETE /identity/v3/projects/6a03c5044d764f21afc9b56b56e9a5d2 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:29.280179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe460c-e717-4e96-98f5-cb952906b79e None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:29.282034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe460c-e717-4e96-98f5-cb952906b79e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:29.282503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbe460c-e717-4e96-98f5-cb952906b79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:29.337316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5cbe460c-e717-4e96-98f5-cb952906b79e None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:29.341353 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5cbe460c-e717-4e96-98f5-cb952906b79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:29.621535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 830/1658] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:29.630704 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d3720-e3aa-4cb8-856a-1056c98c064d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:29.631050 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d3720-e3aa-4cb8-856a-1056c98c064d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:29.631258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d3720-e3aa-4cb8-856a-1056c98c064d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:29.691216 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ad3d3720-e3aa-4cb8-856a-1056c98c064d None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:29.692589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ad3d3720-e3aa-4cb8-856a-1056c98c064d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:29.733231 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 829/1659] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:29.734410 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe460c-e717-4e96-98f5-cb952906b79e None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:29.766877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbe460c-e717-4e96-98f5-cb952906b79e None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:29.784024 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdaafe7-ec20-4034-be5e-f455e6dc5268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:29.784660 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaafe7-ec20-4034-be5e-f455e6dc5268 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:29.784937 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaafe7-ec20-4034-be5e-f455e6dc5268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:29.785213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdaafe7-ec20-4034-be5e-f455e6dc5268 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:29.788825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdaafe7-ec20-4034-be5e-f455e6dc5268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:29.791300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdaafe7-ec20-4034-be5e-f455e6dc5268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:29.794751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdaafe7-ec20-4034-be5e-f455e6dc5268 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:29.829904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 831/1660] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:26:29.833057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d3720-e3aa-4cb8-856a-1056c98c064d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:29.845784 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d3720-e3aa-4cb8-856a-1056c98c064d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:29.859637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109bea72-fa77-4fff-a4d0-632cc839b882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:29.860264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109bea72-fa77-4fff-a4d0-632cc839b882 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:29.860538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109bea72-fa77-4fff-a4d0-632cc839b882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:29.860805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109bea72-fa77-4fff-a4d0-632cc839b882 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:29.861728 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109bea72-fa77-4fff-a4d0-632cc839b882 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:29.864107 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109bea72-fa77-4fff-a4d0-632cc839b882 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:29.965593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdaafe7-ec20-4034-be5e-f455e6dc5268 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:29.965778 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 830/1661] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:26:29 2022] POST /identity/v3/projects => generated 393 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:29.982726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdaafe7-ec20-4034-be5e-f455e6dc5268 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:30.206241 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65863a4a-acbd-4008-9ba2-b8d592d1b12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:30.210058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65863a4a-acbd-4008-9ba2-b8d592d1b12c admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:30.211572 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65863a4a-acbd-4008-9ba2-b8d592d1b12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:30.216103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65863a4a-acbd-4008-9ba2-b8d592d1b12c admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:30.219055 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65863a4a-acbd-4008-9ba2-b8d592d1b12c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:30.231148 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65863a4a-acbd-4008-9ba2-b8d592d1b12c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:30.289684 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 832/1662] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:26:29 2022] POST /identity/v3/users => generated 297 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:30.297361 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109bea72-fa77-4fff-a4d0-632cc839b882 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:30.322310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109bea72-fa77-4fff-a4d0-632cc839b882 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:30.342717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381ac65-f745-43c4-b138-0af3fd1cd0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:30.343529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381ac65-f745-43c4-b138-0af3fd1cd0aa admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:30.360051 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381ac65-f745-43c4-b138-0af3fd1cd0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:30.361716 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381ac65-f745-43c4-b138-0af3fd1cd0aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:30.363393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381ac65-f745-43c4-b138-0af3fd1cd0aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:30.376280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381ac65-f745-43c4-b138-0af3fd1cd0aa admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:30.416999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65863a4a-acbd-4008-9ba2-b8d592d1b12c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:30.420736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 831/1663] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:30.436706 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65863a4a-acbd-4008-9ba2-b8d592d1b12c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:30.458124 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf0994-d7d8-4cae-9547-c5e3c08d2057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:30.464899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf0994-d7d8-4cae-9547-c5e3c08d2057 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:30.465585 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf0994-d7d8-4cae-9547-c5e3c08d2057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:30.466152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf0994-d7d8-4cae-9547-c5e3c08d2057 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:30.467093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf0994-d7d8-4cae-9547-c5e3c08d2057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:30.469844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecf0994-d7d8-4cae-9547-c5e3c08d2057 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:30.517622 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 833/1664] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:30.519667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381ac65-f745-43c4-b138-0af3fd1cd0aa admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:30.595501 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381ac65-f745-43c4-b138-0af3fd1cd0aa admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:30.625911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5a84a-c58f-464c-bf1b-970e88e03726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:30.626969 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5a84a-c58f-464c-bf1b-970e88e03726 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:30.627415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5a84a-c58f-464c-bf1b-970e88e03726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:30.627838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df5a84a-c58f-464c-bf1b-970e88e03726 admin admin] PATH_INFO: `/v3/projects/a2ebd539b9324b349850a23201e1f86b/users/2029340c6d5341b7a793babd4b5adfb6/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:30.700499 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5a84a-c58f-464c-bf1b-970e88e03726 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2ebd539b9324b349850a23201e1f86b, user_id=2029340c6d5341b7a793babd4b5adfb6, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:30.711879 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df5a84a-c58f-464c-bf1b-970e88e03726 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:30.762267 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 832/1665] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:30 2022] PUT /identity/v3/projects/a2ebd539b9324b349850a23201e1f86b/users/2029340c6d5341b7a793babd4b5adfb6/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:30.777246 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccdeef-b7e8-4322-9e6d-3b8cb5ef43fb None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:30.777812 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccdeef-b7e8-4322-9e6d-3b8cb5ef43fb None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:30.778362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccdeef-b7e8-4322-9e6d-3b8cb5ef43fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:30.884514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-baccdeef-b7e8-4322-9e6d-3b8cb5ef43fb None None] MFA Rules not processed for user `2029340c6d5341b7a793babd4b5adfb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:30.888049 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-baccdeef-b7e8-4322-9e6d-3b8cb5ef43fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:31.033392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:31.195593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 834/1666] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:30 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:26:31.200372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5a84a-c58f-464c-bf1b-970e88e03726 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:31.334665 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df5a84a-c58f-464c-bf1b-970e88e03726 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:31.348475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e6d6d-32e5-4e1d-9d87-4e648ea205f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:31.349112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e6d6d-32e5-4e1d-9d87-4e648ea205f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:31.349402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e6d6d-32e5-4e1d-9d87-4e648ea205f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:31.349674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617e6d6d-32e5-4e1d-9d87-4e648ea205f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:31.350372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e6d6d-32e5-4e1d-9d87-4e648ea205f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:31.352491 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617e6d6d-32e5-4e1d-9d87-4e648ea205f9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:31.417975 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 833/1667] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:26:31 2022] POST /identity/v3/projects => generated 391 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:31.419733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baccdeef-b7e8-4322-9e6d-3b8cb5ef43fb None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:31.449700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baccdeef-b7e8-4322-9e6d-3b8cb5ef43fb None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:31.665190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6d78a-bccc-4fe5-aac3-5dcace69c871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:31.666110 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6d78a-bccc-4fe5-aac3-5dcace69c871 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:31.668111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6d78a-bccc-4fe5-aac3-5dcace69c871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:31.668973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af6d78a-bccc-4fe5-aac3-5dcace69c871 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:31.669768 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6d78a-bccc-4fe5-aac3-5dcace69c871 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:31.677221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af6d78a-bccc-4fe5-aac3-5dcace69c871 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:31.745799 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 835/1668] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:26:31 2022] POST /identity/v3/users => generated 302 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:31.750679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e6d6d-32e5-4e1d-9d87-4e648ea205f9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:31.774872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617e6d6d-32e5-4e1d-9d87-4e648ea205f9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:31.794817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dcf51-f118-4a5e-a803-5c619643c2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:31.795841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dcf51-f118-4a5e-a803-5c619643c2eb admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:31.796356 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dcf51-f118-4a5e-a803-5c619643c2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:31.796843 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445dcf51-f118-4a5e-a803-5c619643c2eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:31.797700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445dcf51-f118-4a5e-a803-5c619643c2eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:31.799541 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445dcf51-f118-4a5e-a803-5c619643c2eb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:31.818111 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 834/1669] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:26:31.831843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6d78a-bccc-4fe5-aac3-5dcace69c871 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:31.858242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af6d78a-bccc-4fe5-aac3-5dcace69c871 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:31.902741 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5bf8e-e2db-43f9-99d4-5d0061b93bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:31.904983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5bf8e-e2db-43f9-99d4-5d0061b93bcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:31.908782 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5bf8e-e2db-43f9-99d4-5d0061b93bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:31.910263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d5bf8e-e2db-43f9-99d4-5d0061b93bcc admin admin] PATH_INFO: `/v3/projects/b4caa2d6d59e41e1a11747cc909ecf11/users/33fa52b770a2417b8c96326e87f807e7/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:31.944446 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5bf8e-e2db-43f9-99d4-5d0061b93bcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4caa2d6d59e41e1a11747cc909ecf11, user_id=33fa52b770a2417b8c96326e87f807e7, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:31.948261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d5bf8e-e2db-43f9-99d4-5d0061b93bcc admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:31.985127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dcf51-f118-4a5e-a803-5c619643c2eb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:31.985301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 836/1670] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:31 2022] PUT /identity/v3/projects/b4caa2d6d59e41e1a11747cc909ecf11/users/33fa52b770a2417b8c96326e87f807e7/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:32.004749 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445dcf51-f118-4a5e-a803-5c619643c2eb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:32.076790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:32.176825 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312f3a6-c554-448c-9261-ad7eef51343a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:32.177445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312f3a6-c554-448c-9261-ad7eef51343a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:32.177802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312f3a6-c554-448c-9261-ad7eef51343a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:32.177970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312f3a6-c554-448c-9261-ad7eef51343a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:32.178643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312f3a6-c554-448c-9261-ad7eef51343a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:32.180902 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2312f3a6-c554-448c-9261-ad7eef51343a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:32.213099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5bf8e-e2db-43f9-99d4-5d0061b93bcc admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:32.221546 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 835/1671] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:32.228450 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d5bf8e-e2db-43f9-99d4-5d0061b93bcc admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:32.244349 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b538b-4ac9-4df7-bda8-83707f5903be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:32.245754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b538b-4ac9-4df7-bda8-83707f5903be admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:32.247056 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b538b-4ac9-4df7-bda8-83707f5903be admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:32.247546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575b538b-4ac9-4df7-bda8-83707f5903be admin admin] PATH_INFO: `/v3/projects/b4caa2d6d59e41e1a11747cc909ecf11/users/33fa52b770a2417b8c96326e87f807e7/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:32.251829 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b538b-4ac9-4df7-bda8-83707f5903be admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4caa2d6d59e41e1a11747cc909ecf11, user_id=33fa52b770a2417b8c96326e87f807e7, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:32.257963 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575b538b-4ac9-4df7-bda8-83707f5903be admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:32.289220 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 837/1672] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:32 2022] PUT /identity/v3/projects/b4caa2d6d59e41e1a11747cc909ecf11/users/33fa52b770a2417b8c96326e87f807e7/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:26:32.290255 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341d046-4e36-4a0a-b29b-41d23b615497 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:32.290556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341d046-4e36-4a0a-b29b-41d23b615497 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:32.290832 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341d046-4e36-4a0a-b29b-41d23b615497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:32.351204 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6341d046-4e36-4a0a-b29b-41d23b615497 None None] MFA Rules not processed for user `33fa52b770a2417b8c96326e87f807e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:32.352451 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6341d046-4e36-4a0a-b29b-41d23b615497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:32.574471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 836/1673] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:32 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:32.586420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b538b-4ac9-4df7-bda8-83707f5903be admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:32.588353 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-575b538b-4ac9-4df7-bda8-83707f5903be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:32.714153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575b538b-4ac9-4df7-bda8-83707f5903be admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:32.727601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c0a1c4-3702-4434-ac5c-53b4022f26af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:32.728251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0a1c4-3702-4434-ac5c-53b4022f26af service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:32.728534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0a1c4-3702-4434-ac5c-53b4022f26af service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:32.728810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c0a1c4-3702-4434-ac5c-53b4022f26af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:32.762874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0a1c4-3702-4434-ac5c-53b4022f26af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:32.766449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c0a1c4-3702-4434-ac5c-53b4022f26af service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:32.791398 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 838/1674] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:32 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:32.902355 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341d046-4e36-4a0a-b29b-41d23b615497 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:32.904250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6341d046-4e36-4a0a-b29b-41d23b615497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:32.923090 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6341d046-4e36-4a0a-b29b-41d23b615497 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:32.927575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d43806-fe1f-4c00-bd6e-dd39faa3470a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:32.928639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d43806-fe1f-4c00-bd6e-dd39faa3470a service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:32.929527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d43806-fe1f-4c00-bd6e-dd39faa3470a service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:32.930033 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d43806-fe1f-4c00-bd6e-dd39faa3470a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:33.108608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d43806-fe1f-4c00-bd6e-dd39faa3470a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:33.110976 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d43806-fe1f-4c00-bd6e-dd39faa3470a service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:33.143888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 837/1675] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:33.432557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687b12c-88ad-49d8-96d9-86a095d89baf None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:33.432918 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687b12c-88ad-49d8-96d9-86a095d89baf None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:33.433268 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f687b12c-88ad-49d8-96d9-86a095d89baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:33.501791 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f687b12c-88ad-49d8-96d9-86a095d89baf None None] MFA Rules not processed for user `56535a702f4344aa8a58f11406acf4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:33.503312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f687b12c-88ad-49d8-96d9-86a095d89baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:33.555196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:33.685833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 839/1676] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:26:33.708452 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d43806-fe1f-4c00-bd6e-dd39faa3470a service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:33.711890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b5d43806-fe1f-4c00-bd6e-dd39faa3470a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:33.926171 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:34.018912 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d43806-fe1f-4c00-bd6e-dd39faa3470a service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:34.044519 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03c44b6-4bab-429e-adce-c010f43272ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:34.045493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c44b6-4bab-429e-adce-c010f43272ae service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:34.045966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c44b6-4bab-429e-adce-c010f43272ae service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:34.046493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03c44b6-4bab-429e-adce-c010f43272ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:34.069062 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c44b6-4bab-429e-adce-c010f43272ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:34.070765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03c44b6-4bab-429e-adce-c010f43272ae service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:34.093771 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 838/1677] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:34.118416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46eae1-777a-4ebb-a1f0-5bc192eb86c4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:34.118982 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46eae1-777a-4ebb-a1f0-5bc192eb86c4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:34.119462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d46eae1-777a-4ebb-a1f0-5bc192eb86c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:34.153640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8d46eae1-777a-4ebb-a1f0-5bc192eb86c4 None None] MFA Rules not processed for user `b7a4e434c3c74b239c50da6b5d500c23`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:34.155145 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8d46eae1-777a-4ebb-a1f0-5bc192eb86c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:34.332029 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 840/1678] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:34.345643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669d912-8b82-4ca8-9886-1248bfa9435b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:34.346266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669d912-8b82-4ca8-9886-1248bfa9435b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:34.346814 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669d912-8b82-4ca8-9886-1248bfa9435b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:34.395473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c669d912-8b82-4ca8-9886-1248bfa9435b None None] MFA Rules not processed for user `b7a4e434c3c74b239c50da6b5d500c23`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:34.397812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c669d912-8b82-4ca8-9886-1248bfa9435b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:34.408252 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 839/1679] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:34.420036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d46eae1-777a-4ebb-a1f0-5bc192eb86c4 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:34.421628 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8d46eae1-777a-4ebb-a1f0-5bc192eb86c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:34.441890 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d46eae1-777a-4ebb-a1f0-5bc192eb86c4 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:34.450263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2eae4-8df5-4689-ae39-e6b9f046347b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:34.451352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2eae4-8df5-4689-ae39-e6b9f046347b service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:34.451352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2eae4-8df5-4689-ae39-e6b9f046347b service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:34.451352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2eae4-8df5-4689-ae39-e6b9f046347b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:34.464881 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2eae4-8df5-4689-ae39-e6b9f046347b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:34.466570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2eae4-8df5-4689-ae39-e6b9f046347b service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:34.480992 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 841/1680] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:35.401398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d47cb-16e9-436e-9104-3c7f51ed886f None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:35.401834 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d47cb-16e9-436e-9104-3c7f51ed886f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:35.401834 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d47cb-16e9-436e-9104-3c7f51ed886f None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:35.403377 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 840/1681] 10.222.0.76 () {54 vars in 912 bytes} [Wed Aug 24 15:26:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:35.409802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80aa8d-26d6-4cea-9744-6b5a96049014 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:35.411525 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80aa8d-26d6-4cea-9744-6b5a96049014 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:35.412022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e80aa8d-26d6-4cea-9744-6b5a96049014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:35.467268 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2e80aa8d-26d6-4cea-9744-6b5a96049014 None None] MFA Rules not processed for user `5427bd0ac88042338b956c10cfd07baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:35.468576 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2e80aa8d-26d6-4cea-9744-6b5a96049014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:35.542732 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 842/1682] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 15:26:35 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:35.563284 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d47cb-16e9-436e-9104-3c7f51ed886f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:35.565755 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ae3d47cb-16e9-436e-9104-3c7f51ed886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:35.585572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d47cb-16e9-436e-9104-3c7f51ed886f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:35.589914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7189c9c3-3481-4642-bd75-3a23c5a4429d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:35.590569 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189c9c3-3481-4642-bd75-3a23c5a4429d service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:35.590852 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189c9c3-3481-4642-bd75-3a23c5a4429d service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:35.591125 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7189c9c3-3481-4642-bd75-3a23c5a4429d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:35.599333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189c9c3-3481-4642-bd75-3a23c5a4429d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:35.601014 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7189c9c3-3481-4642-bd75-3a23c5a4429d service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:35.611719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 841/1683] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:26:36.678393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e798b8-0111-4676-af0f-66e8d99dca32 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:36.679142 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e798b8-0111-4676-af0f-66e8d99dca32 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:36.679142 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e798b8-0111-4676-af0f-66e8d99dca32 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:36.680761 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 843/1684] 10.222.0.76 () {54 vars in 961 bytes} [Wed Aug 24 15:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:36.708346 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ab4c9-d866-4c56-ae9b-6aa3f33b3564 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:36.711032 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ab4c9-d866-4c56-ae9b-6aa3f33b3564 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:36.713236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ab4c9-d866-4c56-ae9b-6aa3f33b3564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:36.759496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c24ab4c9-d866-4c56-ae9b-6aa3f33b3564 None None] MFA Rules not processed for user `fb745b98e3ae4723bad44428646f83e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:36.763614 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c24ab4c9-d866-4c56-ae9b-6aa3f33b3564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:36.835213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d1927-866b-4293-92b3-c5dfdca9b879 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:36.835721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d1927-866b-4293-92b3-c5dfdca9b879 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:36.836268 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d1927-866b-4293-92b3-c5dfdca9b879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:36.869190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-317d1927-866b-4293-92b3-c5dfdca9b879 None None] MFA Rules not processed for user `56535a702f4344aa8a58f11406acf4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:36.871514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-317d1927-866b-4293-92b3-c5dfdca9b879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:36.879513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 844/1685] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:36 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:26:36.887980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 842/1686] 10.222.0.76 () {58 vars in 1058 bytes} [Wed Aug 24 15:26:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:26:36.898344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d1927-866b-4293-92b3-c5dfdca9b879 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:36.912498 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d1927-866b-4293-92b3-c5dfdca9b879 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:36.917408 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46728d02-892d-4993-942e-0fd7008cabcd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:36.918049 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46728d02-892d-4993-942e-0fd7008cabcd service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:36.918361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46728d02-892d-4993-942e-0fd7008cabcd service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:36.918634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46728d02-892d-4993-942e-0fd7008cabcd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:36.920483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-46728d02-892d-4993-942e-0fd7008cabcd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:36.983160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46728d02-892d-4993-942e-0fd7008cabcd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:36.992366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46728d02-892d-4993-942e-0fd7008cabcd service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:37.007201 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 845/1687] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:26:41.801523 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ab4c9-d866-4c56-ae9b-6aa3f33b3564 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:41.815485 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ab4c9-d866-4c56-ae9b-6aa3f33b3564 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:41.821552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de5db8-2cd2-446a-a081-209c402bb2d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:41.822346 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de5db8-2cd2-446a-a081-209c402bb2d6 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:41.822536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de5db8-2cd2-446a-a081-209c402bb2d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:41.822791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51de5db8-2cd2-446a-a081-209c402bb2d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:41.825249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-51de5db8-2cd2-446a-a081-209c402bb2d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:41.903759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de5db8-2cd2-446a-a081-209c402bb2d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:41.905208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51de5db8-2cd2-446a-a081-209c402bb2d6 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:41.916364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 843/1688] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:45.269973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a279ec-2047-4b6a-bf49-f42070c45f08 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:45.270545 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a279ec-2047-4b6a-bf49-f42070c45f08 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:45.270545 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a279ec-2047-4b6a-bf49-f42070c45f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:45.292422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:45.328645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-80a279ec-2047-4b6a-bf49-f42070c45f08 None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:45.328645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-80a279ec-2047-4b6a-bf49-f42070c45f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:45.428481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 846/1689] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 15:26:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:45.509529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de5db8-2cd2-446a-a081-209c402bb2d6 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:45.511089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-51de5db8-2cd2-446a-a081-209c402bb2d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:45.566252 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:45.633792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51de5db8-2cd2-446a-a081-209c402bb2d6 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:45.641698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092a916-ca51-4791-b35b-f19ddc1c2693 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:45.642581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092a916-ca51-4791-b35b-f19ddc1c2693 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:45.642997 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092a916-ca51-4791-b35b-f19ddc1c2693 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:45.643389 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092a916-ca51-4791-b35b-f19ddc1c2693 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:45.653980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092a916-ca51-4791-b35b-f19ddc1c2693 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:45.655771 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4092a916-ca51-4791-b35b-f19ddc1c2693 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:45.665393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 844/1690] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:26:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 15:26:55.110697 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a279ec-2047-4b6a-bf49-f42070c45f08 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:55.120700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-80a279ec-2047-4b6a-bf49-f42070c45f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:55.345209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a279ec-2047-4b6a-bf49-f42070c45f08 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:55.350418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b9c7e-da92-472a-b1ee-9ff157c79206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:55.351557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b9c7e-da92-472a-b1ee-9ff157c79206 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:55.351949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b9c7e-da92-472a-b1ee-9ff157c79206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:55.352327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b9c7e-da92-472a-b1ee-9ff157c79206 admin admin] PATH_INFO: `/v3/users/5c0d72599f8b4f0ab924c11f50b8c3f6` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:55.353594 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b9c7e-da92-472a-b1ee-9ff157c79206 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c0d72599f8b4f0ab924c11f50b8c3f6)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:55.355960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b9c7e-da92-472a-b1ee-9ff157c79206 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:55.506927 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f31b9c7e-da92-472a-b1ee-9ff157c79206 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c0d72599f8b4f0ab924c11f50b8c3f6'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:55.537711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f31b9c7e-da92-472a-b1ee-9ff157c79206 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c0d72599f8b4f0ab924c11f50b8c3f6'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:55.573436 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f31b9c7e-da92-472a-b1ee-9ff157c79206 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c0d72599f8b4f0ab924c11f50b8c3f6'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:55.574304 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f31b9c7e-da92-472a-b1ee-9ff157c79206 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c0d72599f8b4f0ab924c11f50b8c3f6'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:55.582493 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f31b9c7e-da92-472a-b1ee-9ff157c79206 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c0d72599f8b4f0ab924c11f50b8c3f6'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:55.595334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 847/1691] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:26:55 2022] DELETE /identity/v3/users/5c0d72599f8b4f0ab924c11f50b8c3f6 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:55.598611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d12433-4f5b-4b00-9c2a-d6b538a65cea None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:55.599508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d12433-4f5b-4b00-9c2a-d6b538a65cea None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:55.599942 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d12433-4f5b-4b00-9c2a-d6b538a65cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:55.659732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-49d12433-4f5b-4b00-9c2a-d6b538a65cea None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:55.661406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-49d12433-4f5b-4b00-9c2a-d6b538a65cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:55.762894 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b9c7e-da92-472a-b1ee-9ff157c79206 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:55.766836 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f31b9c7e-da92-472a-b1ee-9ff157c79206 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:55.775142 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:55.784076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 845/1692] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:55.797939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d12433-4f5b-4b00-9c2a-d6b538a65cea None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:55.799730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-49d12433-4f5b-4b00-9c2a-d6b538a65cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:55.975147 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d12433-4f5b-4b00-9c2a-d6b538a65cea None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:56.006561 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688d3fc6-9bde-40e1-8c71-02704a439241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:56.007707 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d3fc6-9bde-40e1-8c71-02704a439241 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:56.007999 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d3fc6-9bde-40e1-8c71-02704a439241 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:56.008362 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688d3fc6-9bde-40e1-8c71-02704a439241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:56.019219 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b9c7e-da92-472a-b1ee-9ff157c79206 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:56.046627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e637f798-310f-4900-a2dc-4c6315cdf27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:56.047258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637f798-310f-4900-a2dc-4c6315cdf27e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:56.047542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637f798-310f-4900-a2dc-4c6315cdf27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:56.047647 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d3fc6-9bde-40e1-8c71-02704a439241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:56.047856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e637f798-310f-4900-a2dc-4c6315cdf27e admin admin] PATH_INFO: `/v3/users/172faa476a6f4d10b11ad91f828a2ae5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:56.052419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637f798-310f-4900-a2dc-4c6315cdf27e admin admin] RBAC: Authorizing `identity:delete_user(user_id=172faa476a6f4d10b11ad91f828a2ae5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:56.053379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688d3fc6-9bde-40e1-8c71-02704a439241 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:56.054527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e637f798-310f-4900-a2dc-4c6315cdf27e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:56.091021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 846/1693] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:26:56.094713 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731c117-c427-47da-bfc1-fdbd3470bcf1 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:56.098293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731c117-c427-47da-bfc1-fdbd3470bcf1 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:56.099912 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4731c117-c427-47da-bfc1-fdbd3470bcf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:56.177222 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4731c117-c427-47da-bfc1-fdbd3470bcf1 None None] MFA Rules not processed for user `5427bd0ac88042338b956c10cfd07baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:56.179246 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4731c117-c427-47da-bfc1-fdbd3470bcf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:56.204899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 847/1694] 10.222.0.15 () {58 vars in 1015 bytes} [Wed Aug 24 15:26:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:56.267429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731c117-c427-47da-bfc1-fdbd3470bcf1 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:56.274694 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e637f798-310f-4900-a2dc-4c6315cdf27e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '172faa476a6f4d10b11ad91f828a2ae5'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:56.280882 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4731c117-c427-47da-bfc1-fdbd3470bcf1 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:56.288435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae223c7-f94e-407a-bdb3-bb57e8b7391c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:56.289159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae223c7-f94e-407a-bdb3-bb57e8b7391c service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:56.289443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae223c7-f94e-407a-bdb3-bb57e8b7391c service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:56.289744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae223c7-f94e-407a-bdb3-bb57e8b7391c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:56.307482 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e637f798-310f-4900-a2dc-4c6315cdf27e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '172faa476a6f4d10b11ad91f828a2ae5'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:56.314594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae223c7-f94e-407a-bdb3-bb57e8b7391c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:56.317925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae223c7-f94e-407a-bdb3-bb57e8b7391c service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:56.340057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 848/1695] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:56.340729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae223c7-f94e-407a-bdb3-bb57e8b7391c service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:56.364656 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e637f798-310f-4900-a2dc-4c6315cdf27e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '172faa476a6f4d10b11ad91f828a2ae5'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:56.366355 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e637f798-310f-4900-a2dc-4c6315cdf27e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '172faa476a6f4d10b11ad91f828a2ae5'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:56.366648 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae223c7-f94e-407a-bdb3-bb57e8b7391c service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:56.374764 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cbf63-6ea3-4f89-8203-e9bc5b4683e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:56.375559 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cbf63-6ea3-4f89-8203-e9bc5b4683e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:56.375963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cbf63-6ea3-4f89-8203-e9bc5b4683e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:56.376244 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cbf63-6ea3-4f89-8203-e9bc5b4683e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:56.387133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e637f798-310f-4900-a2dc-4c6315cdf27e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '172faa476a6f4d10b11ad91f828a2ae5'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:56.396553 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cbf63-6ea3-4f89-8203-e9bc5b4683e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:56.400786 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cbf63-6ea3-4f89-8203-e9bc5b4683e1 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:56.410682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 848/1696] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:26:55 2022] DELETE /identity/v3/users/172faa476a6f4d10b11ad91f828a2ae5 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:26:56.417655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a94a3a-69d1-4818-87c5-2530c81b8af2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:56.418364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a94a3a-69d1-4818-87c5-2530c81b8af2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:56.419418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a94a3a-69d1-4818-87c5-2530c81b8af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:56.435112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 849/1697] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:56.482628 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-83a94a3a-69d1-4818-87c5-2530c81b8af2 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:56.489546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-83a94a3a-69d1-4818-87c5-2530c81b8af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:56.712210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 849/1698] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:56.749258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cbf63-6ea3-4f89-8203-e9bc5b4683e1 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:56.763249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cbf63-6ea3-4f89-8203-e9bc5b4683e1 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:56.799689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:56.852903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790860e3-23ad-454f-b9d3-a080cb8bc0ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:56.853647 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790860e3-23ad-454f-b9d3-a080cb8bc0ef service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:56.854132 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790860e3-23ad-454f-b9d3-a080cb8bc0ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:56.854570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790860e3-23ad-454f-b9d3-a080cb8bc0ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:56.879435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790860e3-23ad-454f-b9d3-a080cb8bc0ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:56.881955 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790860e3-23ad-454f-b9d3-a080cb8bc0ef service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:56.893812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 850/1699] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:57.159816 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a94a3a-69d1-4818-87c5-2530c81b8af2 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:57.162437 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-83a94a3a-69d1-4818-87c5-2530c81b8af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:57.175128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a94a3a-69d1-4818-87c5-2530c81b8af2 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:57.180838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028424c5-43fc-4860-b2ae-0d40036682fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:57.182454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028424c5-43fc-4860-b2ae-0d40036682fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:57.182985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028424c5-43fc-4860-b2ae-0d40036682fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:57.183316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028424c5-43fc-4860-b2ae-0d40036682fc admin admin] PATH_INFO: `/v3/users/05b1c0c6cdbc49929dc9f0cc14b9315c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:57.184830 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028424c5-43fc-4860-b2ae-0d40036682fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=05b1c0c6cdbc49929dc9f0cc14b9315c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:57.187098 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028424c5-43fc-4860-b2ae-0d40036682fc admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:57.207103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:57.255755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-028424c5-43fc-4860-b2ae-0d40036682fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05b1c0c6cdbc49929dc9f0cc14b9315c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:57.259419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790860e3-23ad-454f-b9d3-a080cb8bc0ef service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:57.260998 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-790860e3-23ad-454f-b9d3-a080cb8bc0ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:57.266773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-028424c5-43fc-4860-b2ae-0d40036682fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05b1c0c6cdbc49929dc9f0cc14b9315c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:57.280199 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790860e3-23ad-454f-b9d3-a080cb8bc0ef service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:57.287371 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571295c-6dcb-42d2-81d2-156dcc3ef6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:57.288188 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571295c-6dcb-42d2-81d2-156dcc3ef6c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:57.288471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571295c-6dcb-42d2-81d2-156dcc3ef6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:57.288766 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571295c-6dcb-42d2-81d2-156dcc3ef6c9 admin admin] PATH_INFO: `/v3/users/2029340c6d5341b7a793babd4b5adfb6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:57.290027 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571295c-6dcb-42d2-81d2-156dcc3ef6c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2029340c6d5341b7a793babd4b5adfb6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:57.290369 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-028424c5-43fc-4860-b2ae-0d40036682fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05b1c0c6cdbc49929dc9f0cc14b9315c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:57.291101 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-028424c5-43fc-4860-b2ae-0d40036682fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05b1c0c6cdbc49929dc9f0cc14b9315c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:57.292173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571295c-6dcb-42d2-81d2-156dcc3ef6c9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:57.300220 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-028424c5-43fc-4860-b2ae-0d40036682fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05b1c0c6cdbc49929dc9f0cc14b9315c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:57.309764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 850/1700] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:26:57 2022] DELETE /identity/v3/users/05b1c0c6cdbc49929dc9f0cc14b9315c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:57.315003 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751da694-b04a-4388-837d-7757d654b2bc None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:57.315338 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751da694-b04a-4388-837d-7757d654b2bc None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:57.315905 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751da694-b04a-4388-837d-7757d654b2bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:57.345531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-751da694-b04a-4388-837d-7757d654b2bc None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:57.348086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-751da694-b04a-4388-837d-7757d654b2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:57.365846 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1571295c-6dcb-42d2-81d2-156dcc3ef6c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2029340c6d5341b7a793babd4b5adfb6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:57.366590 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1571295c-6dcb-42d2-81d2-156dcc3ef6c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2029340c6d5341b7a793babd4b5adfb6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:57.394330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1571295c-6dcb-42d2-81d2-156dcc3ef6c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2029340c6d5341b7a793babd4b5adfb6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:57.412492 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1571295c-6dcb-42d2-81d2-156dcc3ef6c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2029340c6d5341b7a793babd4b5adfb6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:57.414767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1571295c-6dcb-42d2-81d2-156dcc3ef6c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2029340c6d5341b7a793babd4b5adfb6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:57.438618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 851/1701] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:26:57 2022] DELETE /identity/v3/users/2029340c6d5341b7a793babd4b5adfb6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:26:57.439736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89c2dd-4601-4da3-b7f5-31dbdaba65d7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:57.440477 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89c2dd-4601-4da3-b7f5-31dbdaba65d7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:57.440785 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd89c2dd-4601-4da3-b7f5-31dbdaba65d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:57.462932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 851/1702] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:26:57.465367 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cd89c2dd-4601-4da3-b7f5-31dbdaba65d7 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:57.467556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cd89c2dd-4601-4da3-b7f5-31dbdaba65d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:57.478806 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751da694-b04a-4388-837d-7757d654b2bc None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:57.480427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-751da694-b04a-4388-837d-7757d654b2bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:57.497062 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751da694-b04a-4388-837d-7757d654b2bc None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:57.579521 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 852/1703] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:57.581371 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89c2dd-4601-4da3-b7f5-31dbdaba65d7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:57.584210 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cd89c2dd-4601-4da3-b7f5-31dbdaba65d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:57.625549 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32cfb7-1876-4570-827d-356744de95c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:57.626204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32cfb7-1876-4570-827d-356744de95c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:57.626512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32cfb7-1876-4570-827d-356744de95c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:57.626784 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc32cfb7-1876-4570-827d-356744de95c9 admin admin] PATH_INFO: `/v3/projects/b8c52f09126b464fae2407602e740b08` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:57.627945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32cfb7-1876-4570-827d-356744de95c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8c52f09126b464fae2407602e740b08)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:57.629864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:57.630000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc32cfb7-1876-4570-827d-356744de95c9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:57.673518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd89c2dd-4601-4da3-b7f5-31dbdaba65d7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:57.680660 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:57.697963 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fc32cfb7-1876-4570-827d-356744de95c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8c52f09126b464fae2407602e740b08', 'request_id': 'req-fc32cfb7-1876-4570-827d-356744de95c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:57.706242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fc32cfb7-1876-4570-827d-356744de95c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8c52f09126b464fae2407602e740b08', 'request_id': 'req-fc32cfb7-1876-4570-827d-356744de95c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:57.716716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fc32cfb7-1876-4570-827d-356744de95c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8c52f09126b464fae2407602e740b08', 'request_id': 'req-fc32cfb7-1876-4570-827d-356744de95c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:57.735054 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fc32cfb7-1876-4570-827d-356744de95c9 admin admin] The token cache is being invalidate because project b8c52f09126b464fae2407602e740b08 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:26:57.735643 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fc32cfb7-1876-4570-827d-356744de95c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:57.737745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fc32cfb7-1876-4570-827d-356744de95c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:57.744209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a179a47-a8b5-4e85-9ba8-93925b44e2a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:57.744209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a179a47-a8b5-4e85-9ba8-93925b44e2a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:57.744209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a179a47-a8b5-4e85-9ba8-93925b44e2a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:57.744209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a179a47-a8b5-4e85-9ba8-93925b44e2a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:57.744601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4a179a47-a8b5-4e85-9ba8-93925b44e2a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:57.745386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 852/1704] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:26:57 2022] DELETE /identity/v3/projects/b8c52f09126b464fae2407602e740b08 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:57.746934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32cfb7-1876-4570-827d-356744de95c9 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:57.753908 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fc32cfb7-1876-4570-827d-356744de95c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:57.781418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc32cfb7-1876-4570-827d-356744de95c9 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:57.787967 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e42d0-bef8-4d32-b883-a0efd0c2f750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:57.788673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e42d0-bef8-4d32-b883-a0efd0c2f750 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:57.789025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e42d0-bef8-4d32-b883-a0efd0c2f750 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:57.789379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259e42d0-bef8-4d32-b883-a0efd0c2f750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:57.791618 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-259e42d0-bef8-4d32-b883-a0efd0c2f750 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:57.857520 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a179a47-a8b5-4e85-9ba8-93925b44e2a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:57.860150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a179a47-a8b5-4e85-9ba8-93925b44e2a0 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:57.898710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 853/1705] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:57.900375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb45fa3-ab5e-453b-a58d-32d3fff289d0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:57.900671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb45fa3-ab5e-453b-a58d-32d3fff289d0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:57.900950 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb45fa3-ab5e-453b-a58d-32d3fff289d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:57.909077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e42d0-bef8-4d32-b883-a0efd0c2f750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:57.910741 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259e42d0-bef8-4d32-b883-a0efd0c2f750 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:57.958736 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 853/1706] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:57.959983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e42d0-bef8-4d32-b883-a0efd0c2f750 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:57.961604 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-259e42d0-bef8-4d32-b883-a0efd0c2f750 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:57.965366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-dbb45fa3-ab5e-453b-a58d-32d3fff289d0 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:57.966811 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dbb45fa3-ab5e-453b-a58d-32d3fff289d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:57.976296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 854/1707] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:57.977950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259e42d0-bef8-4d32-b883-a0efd0c2f750 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:57.982983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0731bc0e-3dcf-4ac2-ab8e-00b2365b8309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:57.984253 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731bc0e-3dcf-4ac2-ab8e-00b2365b8309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:57.984537 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731bc0e-3dcf-4ac2-ab8e-00b2365b8309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:57.984812 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0731bc0e-3dcf-4ac2-ab8e-00b2365b8309 admin admin] PATH_INFO: `/v3/projects/77bcbb8b4a9b44299ca3ac710c1fdee7` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:57.986139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731bc0e-3dcf-4ac2-ab8e-00b2365b8309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77bcbb8b4a9b44299ca3ac710c1fdee7)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:57.987922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16194c5f-daad-4e62-9510-77c47ed101fd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:57.988212 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16194c5f-daad-4e62-9510-77c47ed101fd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:57.988487 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16194c5f-daad-4e62-9510-77c47ed101fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:57.988579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0731bc0e-3dcf-4ac2-ab8e-00b2365b8309 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:58.010830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-16194c5f-daad-4e62-9510-77c47ed101fd None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:58.013494 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-16194c5f-daad-4e62-9510-77c47ed101fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:58.022955 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 855/1708] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:58.027612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16194c5f-daad-4e62-9510-77c47ed101fd None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:58.040348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16194c5f-daad-4e62-9510-77c47ed101fd None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:58.059186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0731bc0e-3dcf-4ac2-ab8e-00b2365b8309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77bcbb8b4a9b44299ca3ac710c1fdee7', 'request_id': 'req-0731bc0e-3dcf-4ac2-ab8e-00b2365b8309'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:58.068379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0731bc0e-3dcf-4ac2-ab8e-00b2365b8309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77bcbb8b4a9b44299ca3ac710c1fdee7', 'request_id': 'req-0731bc0e-3dcf-4ac2-ab8e-00b2365b8309'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:58.081745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0731bc0e-3dcf-4ac2-ab8e-00b2365b8309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77bcbb8b4a9b44299ca3ac710c1fdee7', 'request_id': 'req-0731bc0e-3dcf-4ac2-ab8e-00b2365b8309'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:58.101192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0731bc0e-3dcf-4ac2-ab8e-00b2365b8309 admin admin] The token cache is being invalidate because project 77bcbb8b4a9b44299ca3ac710c1fdee7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:26:58.101482 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0731bc0e-3dcf-4ac2-ab8e-00b2365b8309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:58.102354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0731bc0e-3dcf-4ac2-ab8e-00b2365b8309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:58.109170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 854/1709] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:26:57 2022] DELETE /identity/v3/projects/77bcbb8b4a9b44299ca3ac710c1fdee7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:58.110912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090d3cf-3675-4ad9-8a61-0d9ae951029f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:58.111254 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090d3cf-3675-4ad9-8a61-0d9ae951029f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:58.111531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7090d3cf-3675-4ad9-8a61-0d9ae951029f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:58.143059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7090d3cf-3675-4ad9-8a61-0d9ae951029f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:58.144353 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7090d3cf-3675-4ad9-8a61-0d9ae951029f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:58.164296 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 855/1710] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:58.164998 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc055961-3356-4ebf-8911-cc0d9138d735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:58.167767 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e498d7-5088-48e3-9287-7616fd9147da None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:58.168199 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e498d7-5088-48e3-9287-7616fd9147da None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:58.168422 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc055961-3356-4ebf-8911-cc0d9138d735 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:58.168523 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e498d7-5088-48e3-9287-7616fd9147da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:58.168741 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc055961-3356-4ebf-8911-cc0d9138d735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:58.169021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc055961-3356-4ebf-8911-cc0d9138d735 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:58.172496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc055961-3356-4ebf-8911-cc0d9138d735 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:58.172828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc055961-3356-4ebf-8911-cc0d9138d735 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:58.178638 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc055961-3356-4ebf-8911-cc0d9138d735 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:58.218106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 856/1711] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:26:58.223121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc055961-3356-4ebf-8911-cc0d9138d735 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:58.224297 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-19e498d7-5088-48e3-9287-7616fd9147da None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:58.225534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-19e498d7-5088-48e3-9287-7616fd9147da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:58.228275 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fc055961-3356-4ebf-8911-cc0d9138d735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:58.244777 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc055961-3356-4ebf-8911-cc0d9138d735 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:58.252739 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625de02f-387e-4e42-b6ea-56850d3a72f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:58.254362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 856/1712] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:58.258445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625de02f-387e-4e42-b6ea-56850d3a72f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:58.258812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625de02f-387e-4e42-b6ea-56850d3a72f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:58.259157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625de02f-387e-4e42-b6ea-56850d3a72f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:58.259786 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e498d7-5088-48e3-9287-7616fd9147da None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:58.260158 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625de02f-387e-4e42-b6ea-56850d3a72f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:58.262241 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625de02f-387e-4e42-b6ea-56850d3a72f8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:58.283772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e498d7-5088-48e3-9287-7616fd9147da None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:58.293799 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:58.297869 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43bc95-a81f-4bec-80b4-98e3f517af26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:58.298626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43bc95-a81f-4bec-80b4-98e3f517af26 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:58.299021 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43bc95-a81f-4bec-80b4-98e3f517af26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:58.299575 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43bc95-a81f-4bec-80b4-98e3f517af26 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:58.302041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43bc95-a81f-4bec-80b4-98e3f517af26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:26:58.302393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43bc95-a81f-4bec-80b4-98e3f517af26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:58.306138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43bc95-a81f-4bec-80b4-98e3f517af26 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:58.308749 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 857/1713] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:26:58 2022] POST /identity/v3/projects => generated 357 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:26:58.313355 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625de02f-387e-4e42-b6ea-56850d3a72f8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:58.355874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 857/1714] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:26:58.360073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43bc95-a81f-4bec-80b4-98e3f517af26 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:58.378016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43bc95-a81f-4bec-80b4-98e3f517af26 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:58.397171 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:58.430538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625de02f-387e-4e42-b6ea-56850d3a72f8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:58.435992 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13199a2c-f2bd-4fc7-9b45-0cdd1487c9f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:58.437121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13199a2c-f2bd-4fc7-9b45-0cdd1487c9f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:58.437400 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13199a2c-f2bd-4fc7-9b45-0cdd1487c9f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:58.437669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13199a2c-f2bd-4fc7-9b45-0cdd1487c9f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:58.438345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13199a2c-f2bd-4fc7-9b45-0cdd1487c9f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:58.440291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13199a2c-f2bd-4fc7-9b45-0cdd1487c9f4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:58.463649 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d939a-2edd-4e9a-9b6d-4c702793fd4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:58.464566 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d939a-2edd-4e9a-9b6d-4c702793fd4d admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:58.465012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d939a-2edd-4e9a-9b6d-4c702793fd4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:58.465448 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09d939a-2edd-4e9a-9b6d-4c702793fd4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:58.466292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d939a-2edd-4e9a-9b6d-4c702793fd4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:58.469413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09d939a-2edd-4e9a-9b6d-4c702793fd4d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:58.470749 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 858/1715] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:26:58 2022] POST /identity/v3/users => generated 279 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:58.474729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13199a2c-f2bd-4fc7-9b45-0cdd1487c9f4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:58.487369 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13199a2c-f2bd-4fc7-9b45-0cdd1487c9f4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:58.502373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714cce8-fd05-49d9-8c56-77869b8a6f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:58.502373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714cce8-fd05-49d9-8c56-77869b8a6f75 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:58.502373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714cce8-fd05-49d9-8c56-77869b8a6f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:58.502373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714cce8-fd05-49d9-8c56-77869b8a6f75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:58.503746 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714cce8-fd05-49d9-8c56-77869b8a6f75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:58.503746 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714cce8-fd05-49d9-8c56-77869b8a6f75 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:58.525405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 859/1716] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:26:58.528089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeeec1-0b63-47bc-833e-4cefe4ea4200 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:58.528471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeeec1-0b63-47bc-833e-4cefe4ea4200 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:58.528945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daeeec1-0b63-47bc-833e-4cefe4ea4200 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:58.532999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 858/1717] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:26:58 2022] POST /identity/v3/projects => generated 375 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:58.534047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d939a-2edd-4e9a-9b6d-4c702793fd4d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:58.538976 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 860/1718] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 15:26:58 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:26:58.540212 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeeec1-0b63-47bc-833e-4cefe4ea4200 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:58.646384 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09d939a-2edd-4e9a-9b6d-4c702793fd4d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:58.647069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daeeec1-0b63-47bc-833e-4cefe4ea4200 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:58.653502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870ec97-6fa2-4e73-8981-e31be3f92b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:58.653934 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1faf64-1a26-434f-a38e-a23ad7a1e45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:58.655959 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870ec97-6fa2-4e73-8981-e31be3f92b8b admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:58.656644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1faf64-1a26-434f-a38e-a23ad7a1e45d admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:58.657078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1faf64-1a26-434f-a38e-a23ad7a1e45d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:58.657377 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1faf64-1a26-434f-a38e-a23ad7a1e45d admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:58.657999 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1faf64-1a26-434f-a38e-a23ad7a1e45d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:58.658480 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870ec97-6fa2-4e73-8981-e31be3f92b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:58.658991 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870ec97-6fa2-4e73-8981-e31be3f92b8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:58.660761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1faf64-1a26-434f-a38e-a23ad7a1e45d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:58.661730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870ec97-6fa2-4e73-8981-e31be3f92b8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:58.664976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870ec97-6fa2-4e73-8981-e31be3f92b8b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:58.692394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 859/1719] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:58.692394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13f552-f68d-4fbf-b1d1-15d97bc6ba46 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:58.692394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13f552-f68d-4fbf-b1d1-15d97bc6ba46 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:58.692394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13f552-f68d-4fbf-b1d1-15d97bc6ba46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:58.704315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 861/1720] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:26:58 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:58.705276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1faf64-1a26-434f-a38e-a23ad7a1e45d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:58.707457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9c1faf64-1a26-434f-a38e-a23ad7a1e45d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:58.723263 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1faf64-1a26-434f-a38e-a23ad7a1e45d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:58.724268 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-1a13f552-f68d-4fbf-b1d1-15d97bc6ba46 None None] MFA Rules not processed for user `8a5fd31a2db944f596912e4f0aa26835`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:58.725426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1a13f552-f68d-4fbf-b1d1-15d97bc6ba46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:58.741468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:58.742130 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:58.742497 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:58.743137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be admin admin] PATH_INFO: `/v3/projects/a2ebd539b9324b349850a23201e1f86b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:58.744082 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2ebd539b9324b349850a23201e1f86b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:58.746054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:58.797847 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2ebd539b9324b349850a23201e1f86b', 'request_id': 'req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:58.804385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2ebd539b9324b349850a23201e1f86b', 'request_id': 'req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:58.806798 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 860/1721] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 15:26:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:58.807694 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13f552-f68d-4fbf-b1d1-15d97bc6ba46 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:58.809646 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=87,keystone:INSERT=7,keystone:DELETE=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:58.816149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2ebd539b9324b349850a23201e1f86b', 'request_id': 'req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:58.826242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13f552-f68d-4fbf-b1d1-15d97bc6ba46 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:58.838488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be admin admin] The token cache is being invalidate because project a2ebd539b9324b349850a23201e1f86b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:26:58.838785 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:58.839472 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:58.841801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 862/1722] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:26:58 2022] DELETE /identity/v3/projects/a2ebd539b9324b349850a23201e1f86b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:58.842622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:58.844297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:58.889342 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:58.924752 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdcf5d-f592-4ba8-bc1d-cfc3564c00be admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:58.930903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d7387-eae7-4246-97fb-9ee9c082ce45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:58.931524 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d7387-eae7-4246-97fb-9ee9c082ce45 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:58.931805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d7387-eae7-4246-97fb-9ee9c082ce45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:58.932669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d7387-eae7-4246-97fb-9ee9c082ce45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:58.932903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d7387-eae7-4246-97fb-9ee9c082ce45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:58.934535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d7387-eae7-4246-97fb-9ee9c082ce45 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:58.935501 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f6374-11c1-4161-918b-20e6fb53a970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:58.935501 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f6374-11c1-4161-918b-20e6fb53a970 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:58.935705 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f6374-11c1-4161-918b-20e6fb53a970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:58.936112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5f6374-11c1-4161-918b-20e6fb53a970 admin admin] PATH_INFO: `/v3/projects/1da088a132824ec79df8504fa1aa6a68/users/e0f476a0b37c4d229777ac4207c74206/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:58.948691 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 863/1723] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:58.949720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d7387-eae7-4246-97fb-9ee9c082ce45 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:58.952385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-502d7387-eae7-4246-97fb-9ee9c082ce45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:58.955486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f6374-11c1-4161-918b-20e6fb53a970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1da088a132824ec79df8504fa1aa6a68, user_id=e0f476a0b37c4d229777ac4207c74206, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:58.958383 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5f6374-11c1-4161-918b-20e6fb53a970 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:58.968211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d7387-eae7-4246-97fb-9ee9c082ce45 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:58.973202 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:58.974487 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:58.974922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:58.975046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36 admin admin] PATH_INFO: `/v3/projects/63480381e3f44c7fbb93a62aa5b09800` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:58.976197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63480381e3f44c7fbb93a62aa5b09800)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:58.978154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:58.985978 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 861/1724] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:58 2022] PUT /identity/v3/projects/1da088a132824ec79df8504fa1aa6a68/users/e0f476a0b37c4d229777ac4207c74206/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:58.986922 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f6374-11c1-4161-918b-20e6fb53a970 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:58.988602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ec5f6374-11c1-4161-918b-20e6fb53a970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:59.065592 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63480381e3f44c7fbb93a62aa5b09800', 'request_id': 'req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.072784 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63480381e3f44c7fbb93a62aa5b09800', 'request_id': 'req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.081431 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63480381e3f44c7fbb93a62aa5b09800', 'request_id': 'req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.089789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5f6374-11c1-4161-918b-20e6fb53a970 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:59.094661 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36 admin admin] The token cache is being invalidate because project 63480381e3f44c7fbb93a62aa5b09800 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:26:59.094948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.095613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.097135 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 864/1725] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:26:58 2022] DELETE /identity/v3/projects/63480381e3f44c7fbb93a62aa5b09800 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:59.099236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:59.101169 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:59.161337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfb45c-c8ee-4c62-8a8e-0155ccc610e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:59.162618 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfb45c-c8ee-4c62-8a8e-0155ccc610e4 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:59.163035 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfb45c-c8ee-4c62-8a8e-0155ccc610e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:59.163308 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbfb45c-c8ee-4c62-8a8e-0155ccc610e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:59.165120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-edbfb45c-c8ee-4c62-8a8e-0155ccc610e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:59.174505 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14be2d0c-79d0-4c3a-85d1-a8c599f54a36 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:59.177784 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfb45c-c8ee-4c62-8a8e-0155ccc610e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:59.179441 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbfb45c-c8ee-4c62-8a8e-0155ccc610e4 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:59.179579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b464686-bc7a-4f98-af12-40722cea2361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:59.180218 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b464686-bc7a-4f98-af12-40722cea2361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:59.180492 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b464686-bc7a-4f98-af12-40722cea2361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:59.180760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b464686-bc7a-4f98-af12-40722cea2361 admin admin] PATH_INFO: `/v3/users/33fa52b770a2417b8c96326e87f807e7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:59.181875 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b464686-bc7a-4f98-af12-40722cea2361 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33fa52b770a2417b8c96326e87f807e7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:59.183804 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b464686-bc7a-4f98-af12-40722cea2361 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:59.190026 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 862/1726] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:26:59.190915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfb45c-c8ee-4c62-8a8e-0155ccc610e4 service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:59.192546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-edbfb45c-c8ee-4c62-8a8e-0155ccc610e4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:59.200288 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:59.205941 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbfb45c-c8ee-4c62-8a8e-0155ccc610e4 service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:59.211442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ebddc-6e2e-45fc-96c4-e1a6105244fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:59.212053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ebddc-6e2e-45fc-96c4-e1a6105244fb admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:59.212317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ebddc-6e2e-45fc-96c4-e1a6105244fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:59.212585 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ebddc-6e2e-45fc-96c4-e1a6105244fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:59.213229 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ebddc-6e2e-45fc-96c4-e1a6105244fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:59.214945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21ebddc-6e2e-45fc-96c4-e1a6105244fb admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:59.230375 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 863/1727] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:26:59.232238 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926b21b-3693-483f-a458-7e731c380a09 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:59.232524 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926b21b-3693-483f-a458-7e731c380a09 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:59.232797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926b21b-3693-483f-a458-7e731c380a09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:59.258763 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1b464686-bc7a-4f98-af12-40722cea2361 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33fa52b770a2417b8c96326e87f807e7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.259454 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1b464686-bc7a-4f98-af12-40722cea2361 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33fa52b770a2417b8c96326e87f807e7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.282466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1b464686-bc7a-4f98-af12-40722cea2361 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33fa52b770a2417b8c96326e87f807e7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.289851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0926b21b-3693-483f-a458-7e731c380a09 None None] MFA Rules not processed for user `e0f476a0b37c4d229777ac4207c74206`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:59.291970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0926b21b-3693-483f-a458-7e731c380a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:59.299482 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1b464686-bc7a-4f98-af12-40722cea2361 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33fa52b770a2417b8c96326e87f807e7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.300637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1b464686-bc7a-4f98-af12-40722cea2361 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33fa52b770a2417b8c96326e87f807e7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.314967 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:59.319748 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 865/1728] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:26:59 2022] DELETE /identity/v3/users/33fa52b770a2417b8c96326e87f807e7 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:59.320811 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b464686-bc7a-4f98-af12-40722cea2361 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:59.323549 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1b464686-bc7a-4f98-af12-40722cea2361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:59.390282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 864/1729] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:59 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:59.391220 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926b21b-3693-483f-a458-7e731c380a09 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:59.392753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0926b21b-3693-483f-a458-7e731c380a09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:59.411091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0926b21b-3693-483f-a458-7e731c380a09 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:59.420378 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b464686-bc7a-4f98-af12-40722cea2361 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:59.421600 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35db8c1-1913-47bc-8270-27d6915bfab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:59.422601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35db8c1-1913-47bc-8270-27d6915bfab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:59.423302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35db8c1-1913-47bc-8270-27d6915bfab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:59.423733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35db8c1-1913-47bc-8270-27d6915bfab5 admin admin] PATH_INFO: `/v3/projects/fbf43b4e2ecb4fa387ad81a4a3a0d7ef/users/23d64c421a554761babc5522164b7065/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:59.426762 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4babd27c-557a-468b-895b-0e3be9b98458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:59.427640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babd27c-557a-468b-895b-0e3be9b98458 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:59.427919 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babd27c-557a-468b-895b-0e3be9b98458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:59.428191 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4babd27c-557a-468b-895b-0e3be9b98458 admin admin] PATH_INFO: `/v3/users/c86122cbced04d2da61e3067964a211f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:59.430467 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4babd27c-557a-468b-895b-0e3be9b98458 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c86122cbced04d2da61e3067964a211f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:59.432511 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4babd27c-557a-468b-895b-0e3be9b98458 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:59.442885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35db8c1-1913-47bc-8270-27d6915bfab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbf43b4e2ecb4fa387ad81a4a3a0d7ef, user_id=23d64c421a554761babc5522164b7065, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:59.445700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35db8c1-1913-47bc-8270-27d6915bfab5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:59.465269 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 865/1730] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:26:59 2022] PUT /identity/v3/projects/fbf43b4e2ecb4fa387ad81a4a3a0d7ef/users/23d64c421a554761babc5522164b7065/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:59.466212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35db8c1-1913-47bc-8270-27d6915bfab5 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:59.471849 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f35db8c1-1913-47bc-8270-27d6915bfab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:59.508112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4babd27c-557a-468b-895b-0e3be9b98458 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c86122cbced04d2da61e3067964a211f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.510041 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4babd27c-557a-468b-895b-0e3be9b98458 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c86122cbced04d2da61e3067964a211f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.533453 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4babd27c-557a-468b-895b-0e3be9b98458 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c86122cbced04d2da61e3067964a211f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.545478 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4babd27c-557a-468b-895b-0e3be9b98458 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c86122cbced04d2da61e3067964a211f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.546324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4babd27c-557a-468b-895b-0e3be9b98458 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c86122cbced04d2da61e3067964a211f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.563765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 866/1731] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:26:59 2022] DELETE /identity/v3/users/c86122cbced04d2da61e3067964a211f => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:59.565078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af20cc-d830-4e4a-846e-ea2472a39fa7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:59.565359 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af20cc-d830-4e4a-846e-ea2472a39fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:59.565627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71af20cc-d830-4e4a-846e-ea2472a39fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:59.589686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35db8c1-1913-47bc-8270-27d6915bfab5 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:59.591659 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=20,keystone:SELECT=78,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:59.613246 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-71af20cc-d830-4e4a-846e-ea2472a39fa7 None None] MFA Rules not processed for user `23d64c421a554761babc5522164b7065`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:26:59.614611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-71af20cc-d830-4e4a-846e-ea2472a39fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:59.665479 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02bcb3d-6573-417b-9e09-c37c2b5cd963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:59.666051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bcb3d-6573-417b-9e09-c37c2b5cd963 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:59.666482 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bcb3d-6573-417b-9e09-c37c2b5cd963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:59.666838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02bcb3d-6573-417b-9e09-c37c2b5cd963 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:59.667613 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02bcb3d-6573-417b-9e09-c37c2b5cd963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:59.670240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02bcb3d-6573-417b-9e09-c37c2b5cd963 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:59.707160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 866/1732] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:26:59 2022] POST /identity/v3/projects => generated 355 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:59.708057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02bcb3d-6573-417b-9e09-c37c2b5cd963 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:59.709838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d02bcb3d-6573-417b-9e09-c37c2b5cd963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:59.739230 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:26:59.750556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 867/1733] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:26:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:26:59.750556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af20cc-d830-4e4a-846e-ea2472a39fa7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:59.750556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-71af20cc-d830-4e4a-846e-ea2472a39fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:59.763984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71af20cc-d830-4e4a-846e-ea2472a39fa7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:59.822281 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02bcb3d-6573-417b-9e09-c37c2b5cd963 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:59.825546 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d791b-19d1-40c9-9f7f-c21fe52a3bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:59.826172 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d791b-19d1-40c9-9f7f-c21fe52a3bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:59.826501 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d791b-19d1-40c9-9f7f-c21fe52a3bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:59.826774 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d791b-19d1-40c9-9f7f-c21fe52a3bb5 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:59.827439 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d791b-19d1-40c9-9f7f-c21fe52a3bb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:59.828584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfd1c4-6b1d-4766-ba64-e25e79887f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:26:59.829223 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfd1c4-6b1d-4766-ba64-e25e79887f08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:26:59.829414 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d791b-19d1-40c9-9f7f-c21fe52a3bb5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:59.829566 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfd1c4-6b1d-4766-ba64-e25e79887f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:26:59.829785 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfd1c4-6b1d-4766-ba64-e25e79887f08 admin admin] PATH_INFO: `/v3/projects/b4caa2d6d59e41e1a11747cc909ecf11` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:26:59.830921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dfd1c4-6b1d-4766-ba64-e25e79887f08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4caa2d6d59e41e1a11747cc909ecf11)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:26:59.833090 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dfd1c4-6b1d-4766-ba64-e25e79887f08 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:26:59.869037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 868/1734] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:26:59 2022] POST /identity/v3/users => generated 284 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:26:59.872371 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d791b-19d1-40c9-9f7f-c21fe52a3bb5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:59.873231 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-592d791b-19d1-40c9-9f7f-c21fe52a3bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:59.896905 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-35dfd1c4-6b1d-4766-ba64-e25e79887f08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4caa2d6d59e41e1a11747cc909ecf11', 'request_id': 'req-35dfd1c4-6b1d-4766-ba64-e25e79887f08'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.903533 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-35dfd1c4-6b1d-4766-ba64-e25e79887f08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4caa2d6d59e41e1a11747cc909ecf11', 'request_id': 'req-35dfd1c4-6b1d-4766-ba64-e25e79887f08'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.914050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-35dfd1c4-6b1d-4766-ba64-e25e79887f08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4caa2d6d59e41e1a11747cc909ecf11', 'request_id': 'req-35dfd1c4-6b1d-4766-ba64-e25e79887f08'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.932045 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-35dfd1c4-6b1d-4766-ba64-e25e79887f08 admin admin] The token cache is being invalidate because project b4caa2d6d59e41e1a11747cc909ecf11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:26:59.932539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-35dfd1c4-6b1d-4766-ba64-e25e79887f08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.933369 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-35dfd1c4-6b1d-4766-ba64-e25e79887f08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:26:59.935113 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 867/1735] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:26:59 2022] DELETE /identity/v3/projects/b4caa2d6d59e41e1a11747cc909ecf11 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:26:59.935956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfd1c4-6b1d-4766-ba64-e25e79887f08 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:26:59.937445 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-35dfd1c4-6b1d-4766-ba64-e25e79887f08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:26:59.964302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d791b-19d1-40c9-9f7f-c21fe52a3bb5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:26:59.996881 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:00.033174 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfd1c4-6b1d-4766-ba64-e25e79887f08 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:00.037868 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930a1e3-acc3-42d2-bb68-aaf400ed22fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:00.038497 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930a1e3-acc3-42d2-bb68-aaf400ed22fa admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.038773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930a1e3-acc3-42d2-bb68-aaf400ed22fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.039039 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930a1e3-acc3-42d2-bb68-aaf400ed22fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.039680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930a1e3-acc3-42d2-bb68-aaf400ed22fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:00.041301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5930a1e3-acc3-42d2-bb68-aaf400ed22fa admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:00.051841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 868/1736] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:26:59 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:00.053905 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0beef0-80e4-4a7a-944a-8d23aed87a8f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:00.054565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0beef0-80e4-4a7a-944a-8d23aed87a8f service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.054945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0beef0-80e4-4a7a-944a-8d23aed87a8f service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.055230 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0beef0-80e4-4a7a-944a-8d23aed87a8f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.055976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc6a31-653b-401e-ae6e-0441cd51f3f2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.056452 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc6a31-653b-401e-ae6e-0441cd51f3f2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.056731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cc6a31-653b-401e-ae6e-0441cd51f3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.057444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6b0beef0-80e4-4a7a-944a-8d23aed87a8f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:00.080355 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b9cc6a31-653b-401e-ae6e-0441cd51f3f2 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:00.081643 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b9cc6a31-653b-401e-ae6e-0441cd51f3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:00.097426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 869/1737] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:00.097426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc6a31-653b-401e-ae6e-0441cd51f3f2 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:00.097426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b9cc6a31-653b-401e-ae6e-0441cd51f3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:00.104857 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cc6a31-653b-401e-ae6e-0441cd51f3f2 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:00.111015 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15429042-029b-4480-8b9d-5534c37a5a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:00.111654 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15429042-029b-4480-8b9d-5534c37a5a4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.111937 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15429042-029b-4480-8b9d-5534c37a5a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.112206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15429042-029b-4480-8b9d-5534c37a5a4f admin admin] PATH_INFO: `/v3/projects/3fffff519d354bd6a8ee7250b8cc3b87` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.113333 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15429042-029b-4480-8b9d-5534c37a5a4f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fffff519d354bd6a8ee7250b8cc3b87)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:00.115389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15429042-029b-4480-8b9d-5534c37a5a4f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:00.132255 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0beef0-80e4-4a7a-944a-8d23aed87a8f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:00.133860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0beef0-80e4-4a7a-944a-8d23aed87a8f service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:00.144021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 869/1738] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:26:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:00.144942 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0beef0-80e4-4a7a-944a-8d23aed87a8f service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:00.161969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:00.165830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0beef0-80e4-4a7a-944a-8d23aed87a8f service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:00.178563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-15429042-029b-4480-8b9d-5534c37a5a4f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fffff519d354bd6a8ee7250b8cc3b87', 'request_id': 'req-15429042-029b-4480-8b9d-5534c37a5a4f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:00.194928 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-15429042-029b-4480-8b9d-5534c37a5a4f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fffff519d354bd6a8ee7250b8cc3b87', 'request_id': 'req-15429042-029b-4480-8b9d-5534c37a5a4f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:00.213541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-15429042-029b-4480-8b9d-5534c37a5a4f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fffff519d354bd6a8ee7250b8cc3b87', 'request_id': 'req-15429042-029b-4480-8b9d-5534c37a5a4f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:00.245012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-15429042-029b-4480-8b9d-5534c37a5a4f admin admin] The token cache is being invalidate because project 3fffff519d354bd6a8ee7250b8cc3b87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:00.245307 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-15429042-029b-4480-8b9d-5534c37a5a4f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:00.246962 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-15429042-029b-4480-8b9d-5534c37a5a4f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:00.253557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 870/1739] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:00 2022] DELETE /identity/v3/projects/3fffff519d354bd6a8ee7250b8cc3b87 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:00.253557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c80f9a-87e8-4b74-923e-7fe6fa0c3e26 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.253557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c80f9a-87e8-4b74-923e-7fe6fa0c3e26 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.253557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c80f9a-87e8-4b74-923e-7fe6fa0c3e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.279923 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b9c80f9a-87e8-4b74-923e-7fe6fa0c3e26 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:00.279923 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b9c80f9a-87e8-4b74-923e-7fe6fa0c3e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:00.311232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d86afeb-9127-4e64-96fe-76f3e87f4746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:00.313018 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86afeb-9127-4e64-96fe-76f3e87f4746 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.313451 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86afeb-9127-4e64-96fe-76f3e87f4746 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.313996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d86afeb-9127-4e64-96fe-76f3e87f4746 admin admin] PATH_INFO: `/v3/projects/1c2534f7d77541dc8798b47ae613863a/users/654cfaaed148401abf77df26ef22a85e/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.333374 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86afeb-9127-4e64-96fe-76f3e87f4746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2534f7d77541dc8798b47ae613863a, user_id=654cfaaed148401abf77df26ef22a85e, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:00.336242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d86afeb-9127-4e64-96fe-76f3e87f4746 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:00.358257 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 870/1740] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:00 2022] PUT /identity/v3/projects/1c2534f7d77541dc8798b47ae613863a/users/654cfaaed148401abf77df26ef22a85e/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:00.359500 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cfcdb-b56b-48fd-b726-98aa898da53e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.359806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cfcdb-b56b-48fd-b726-98aa898da53e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.360109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6cfcdb-b56b-48fd-b726-98aa898da53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.367693 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 871/1741] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:00.369112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c80f9a-87e8-4b74-923e-7fe6fa0c3e26 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:00.372050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b9c80f9a-87e8-4b74-923e-7fe6fa0c3e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:00.387130 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9d6cfcdb-b56b-48fd-b726-98aa898da53e None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:00.387565 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c80f9a-87e8-4b74-923e-7fe6fa0c3e26 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:00.388330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9d6cfcdb-b56b-48fd-b726-98aa898da53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:00.450429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:00.466529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8774f-8c20-45c9-a50d-b1f88056d58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:00.467277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8774f-8c20-45c9-a50d-b1f88056d58d admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.467622 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8774f-8c20-45c9-a50d-b1f88056d58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.468490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8774f-8c20-45c9-a50d-b1f88056d58d admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.469203 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8774f-8c20-45c9-a50d-b1f88056d58d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:00.470959 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8774f-8c20-45c9-a50d-b1f88056d58d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:00.482427 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 871/1742] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:00.482427 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6cfcdb-b56b-48fd-b726-98aa898da53e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:00.482704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 872/1743] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:00.482704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8774f-8c20-45c9-a50d-b1f88056d58d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:00.485994 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-64e8774f-8c20-45c9-a50d-b1f88056d58d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:00.500459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6cfcdb-b56b-48fd-b726-98aa898da53e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:00.508264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8774f-8c20-45c9-a50d-b1f88056d58d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:00.509489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a869d969-f6e4-4a92-a283-c8866fc3ee31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:00.510124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869d969-f6e4-4a92-a283-c8866fc3ee31 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.510438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869d969-f6e4-4a92-a283-c8866fc3ee31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.510709 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a869d969-f6e4-4a92-a283-c8866fc3ee31 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.511409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869d969-f6e4-4a92-a283-c8866fc3ee31 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:00.511718 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869d969-f6e4-4a92-a283-c8866fc3ee31 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:00.513558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a869d969-f6e4-4a92-a283-c8866fc3ee31 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:00.513677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869ee6a-8a21-4ed0-a75c-783f15a364a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:00.515938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869ee6a-8a21-4ed0-a75c-783f15a364a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.516387 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869ee6a-8a21-4ed0-a75c-783f15a364a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.516933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869ee6a-8a21-4ed0-a75c-783f15a364a9 admin admin] PATH_INFO: `/v3/users/23d64c421a554761babc5522164b7065` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.518609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869ee6a-8a21-4ed0-a75c-783f15a364a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23d64c421a554761babc5522164b7065)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:00.520696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869ee6a-8a21-4ed0-a75c-783f15a364a9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:00.529388 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 872/1744] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:00.530773 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2123e5-b280-4b53-b923-30818057677e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.531060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2123e5-b280-4b53-b923-30818057677e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.531335 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2123e5-b280-4b53-b923-30818057677e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.557420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ac2123e5-b280-4b53-b923-30818057677e None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:00.558734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ac2123e5-b280-4b53-b923-30818057677e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:00.574514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 873/1745] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:00.575412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2123e5-b280-4b53-b923-30818057677e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:00.586313 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2123e5-b280-4b53-b923-30818057677e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:00.592041 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5478eee-24a0-46a6-b6eb-63551b626483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:00.593372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5478eee-24a0-46a6-b6eb-63551b626483 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.593623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5478eee-24a0-46a6-b6eb-63551b626483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.593903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5478eee-24a0-46a6-b6eb-63551b626483 admin admin] PATH_INFO: `/v3/projects/1c2534f7d77541dc8798b47ae613863a/users/654cfaaed148401abf77df26ef22a85e/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.596110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5478eee-24a0-46a6-b6eb-63551b626483 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c2534f7d77541dc8798b47ae613863a, user_id=654cfaaed148401abf77df26ef22a85e, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:00.599718 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5478eee-24a0-46a6-b6eb-63551b626483 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:00.599871 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c869ee6a-8a21-4ed0-a75c-783f15a364a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23d64c421a554761babc5522164b7065'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:00.613096 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 874/1746] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:00 2022] PUT /identity/v3/projects/1c2534f7d77541dc8798b47ae613863a/users/654cfaaed148401abf77df26ef22a85e/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:00.613959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5478eee-24a0-46a6-b6eb-63551b626483 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:00.616818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c869ee6a-8a21-4ed0-a75c-783f15a364a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23d64c421a554761babc5522164b7065'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:00.623701 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:00.653547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c869ee6a-8a21-4ed0-a75c-783f15a364a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23d64c421a554761babc5522164b7065'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:00.654564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c869ee6a-8a21-4ed0-a75c-783f15a364a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23d64c421a554761babc5522164b7065'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:00.679378 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c869ee6a-8a21-4ed0-a75c-783f15a364a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23d64c421a554761babc5522164b7065'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:00.686784 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 873/1747] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:00 2022] DELETE /identity/v3/users/23d64c421a554761babc5522164b7065 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:00.687954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869ee6a-8a21-4ed0-a75c-783f15a364a9 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:00.699569 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c869ee6a-8a21-4ed0-a75c-783f15a364a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:00.726916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5478eee-24a0-46a6-b6eb-63551b626483 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:00.775044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869ee6a-8a21-4ed0-a75c-783f15a364a9 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:00.780209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e075fb-7b58-46a4-9208-81365c967899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:00.781076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e075fb-7b58-46a4-9208-81365c967899 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.781642 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e075fb-7b58-46a4-9208-81365c967899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.783211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e075fb-7b58-46a4-9208-81365c967899 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.783334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e075fb-7b58-46a4-9208-81365c967899 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:00.783580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e075fb-7b58-46a4-9208-81365c967899 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:00.785235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e075fb-7b58-46a4-9208-81365c967899 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:00.791265 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff81b7c-b3b4-4a56-bbd8-11f6eaf47954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:00.791871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff81b7c-b3b4-4a56-bbd8-11f6eaf47954 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.792150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff81b7c-b3b4-4a56-bbd8-11f6eaf47954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.792423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff81b7c-b3b4-4a56-bbd8-11f6eaf47954 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.793038 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff81b7c-b3b4-4a56-bbd8-11f6eaf47954 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:00.794969 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff81b7c-b3b4-4a56-bbd8-11f6eaf47954 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:00.804701 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 874/1748] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:00.804701 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4dacf-f433-4824-b440-38606688227f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.804701 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4dacf-f433-4824-b440-38606688227f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.804701 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4dacf-f433-4824-b440-38606688227f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.828923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 875/1749] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:00 2022] POST /identity/v3/projects => generated 383 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:00.833897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97e1ea-cd60-456a-9729-d9cad54724f5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.834257 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97e1ea-cd60-456a-9729-d9cad54724f5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.834535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97e1ea-cd60-456a-9729-d9cad54724f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.837953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f3b4dacf-f433-4824-b440-38606688227f None None] MFA Rules not processed for user `654cfaaed148401abf77df26ef22a85e`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:00.839273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f3b4dacf-f433-4824-b440-38606688227f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:00.861877 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6e97e1ea-cd60-456a-9729-d9cad54724f5 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:00.863135 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6e97e1ea-cd60-456a-9729-d9cad54724f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:00.927198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:00.933166 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 875/1750] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:00.933850 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4dacf-f433-4824-b440-38606688227f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:00.951978 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 876/1751] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:00.952842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97e1ea-cd60-456a-9729-d9cad54724f5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:00.954134 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6e97e1ea-cd60-456a-9729-d9cad54724f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:00.966778 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97e1ea-cd60-456a-9729-d9cad54724f5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:00.971792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b89c5b-f9f0-4472-8edf-1a39fad16ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:00.972403 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b89c5b-f9f0-4472-8edf-1a39fad16ac5 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:00.972681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b89c5b-f9f0-4472-8edf-1a39fad16ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:00.972949 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b89c5b-f9f0-4472-8edf-1a39fad16ac5 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:00.973568 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b89c5b-f9f0-4472-8edf-1a39fad16ac5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:00.975500 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b89c5b-f9f0-4472-8edf-1a39fad16ac5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:00.994129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:00.997462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 877/1752] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:00 2022] POST /identity/v3/users => generated 292 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:01.001188 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b89c5b-f9f0-4472-8edf-1a39fad16ac5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:01.004419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-25b89c5b-f9f0-4472-8edf-1a39fad16ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:01.008328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4dacf-f433-4824-b440-38606688227f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:01.013055 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441cbc7-5ced-4fbf-b791-2eec5784e360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:01.013664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441cbc7-5ced-4fbf-b791-2eec5784e360 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:01.013947 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441cbc7-5ced-4fbf-b791-2eec5784e360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:01.014252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a441cbc7-5ced-4fbf-b791-2eec5784e360 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:01.014894 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441cbc7-5ced-4fbf-b791-2eec5784e360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:01.016843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a441cbc7-5ced-4fbf-b791-2eec5784e360 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:01.065913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 876/1753] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:00 2022] POST /identity/v3/projects => generated 399 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:01.066799 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441cbc7-5ced-4fbf-b791-2eec5784e360 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:01.067351 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b89c5b-f9f0-4472-8edf-1a39fad16ac5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:01.070657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a441cbc7-5ced-4fbf-b791-2eec5784e360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:01.117731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f76cbc-70a1-4aa5-8cef-2490430f4eae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:01.118472 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f76cbc-70a1-4aa5-8cef-2490430f4eae service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:01.118821 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f76cbc-70a1-4aa5-8cef-2490430f4eae service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:01.119161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f76cbc-70a1-4aa5-8cef-2490430f4eae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:01.142271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a441cbc7-5ced-4fbf-b791-2eec5784e360 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:01.146564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a7c90-f710-4e13-83f9-10f8be80f823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:01.147165 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a7c90-f710-4e13-83f9-10f8be80f823 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:01.147439 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a7c90-f710-4e13-83f9-10f8be80f823 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:01.147708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62a7c90-f710-4e13-83f9-10f8be80f823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:01.187232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f76cbc-70a1-4aa5-8cef-2490430f4eae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:01.188696 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f76cbc-70a1-4aa5-8cef-2490430f4eae service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:01.199031 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 878/1754] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:00 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:01.199779 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f76cbc-70a1-4aa5-8cef-2490430f4eae service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:01.212295 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a7c90-f710-4e13-83f9-10f8be80f823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:01.213739 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62a7c90-f710-4e13-83f9-10f8be80f823 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:01.226591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 877/1755] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:01.227419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a7c90-f710-4e13-83f9-10f8be80f823 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:01.239139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62a7c90-f710-4e13-83f9-10f8be80f823 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:01.244286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc37f0-7bf8-4c2f-a217-a1e3de2d7070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:01.244939 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc37f0-7bf8-4c2f-a217-a1e3de2d7070 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:01.245219 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc37f0-7bf8-4c2f-a217-a1e3de2d7070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:01.245609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc37f0-7bf8-4c2f-a217-a1e3de2d7070 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:01.246140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc37f0-7bf8-4c2f-a217-a1e3de2d7070 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:01.248097 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc37f0-7bf8-4c2f-a217-a1e3de2d7070 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:01.248317 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:01.282482 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 878/1756] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:01 2022] POST /identity/v3/users => generated 300 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:01.283733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc37f0-7bf8-4c2f-a217-a1e3de2d7070 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:01.289852 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f76cbc-70a1-4aa5-8cef-2490430f4eae service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:01.295805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168cdfa-ae24-438b-86a9-0d2e6d4f4773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:01.297187 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168cdfa-ae24-438b-86a9-0d2e6d4f4773 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:01.297778 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168cdfa-ae24-438b-86a9-0d2e6d4f4773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:01.298249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8168cdfa-ae24-438b-86a9-0d2e6d4f4773 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:01.299182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8168cdfa-ae24-438b-86a9-0d2e6d4f4773 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:01.301375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8168cdfa-ae24-438b-86a9-0d2e6d4f4773 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:01.302145 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc37f0-7bf8-4c2f-a217-a1e3de2d7070 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:01.318393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 879/1757] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:01.319638 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168cdfa-ae24-438b-86a9-0d2e6d4f4773 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:01.333266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8168cdfa-ae24-438b-86a9-0d2e6d4f4773 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:01.337727 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c045f-e876-445e-a34e-666c98682062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:01.339757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c045f-e876-445e-a34e-666c98682062 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:01.340880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c045f-e876-445e-a34e-666c98682062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:01.341716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180c045f-e876-445e-a34e-666c98682062 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:01.344378 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a43d0-b563-4c8c-8070-f107057d5203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:01.344817 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c045f-e876-445e-a34e-666c98682062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:01.347003 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180c045f-e876-445e-a34e-666c98682062 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:01.347273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a43d0-b563-4c8c-8070-f107057d5203 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:01.347621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a43d0-b563-4c8c-8070-f107057d5203 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:01.347934 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2a43d0-b563-4c8c-8070-f107057d5203 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:01.348903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a43d0-b563-4c8c-8070-f107057d5203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:01.351957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2a43d0-b563-4c8c-8070-f107057d5203 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:01.373347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 879/1758] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:01.374142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 880/1759] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:01.374577 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c045f-e876-445e-a34e-666c98682062 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:01.374808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a43d0-b563-4c8c-8070-f107057d5203 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:01.387332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2a43d0-b563-4c8c-8070-f107057d5203 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:01.390111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180c045f-e876-445e-a34e-666c98682062 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:01.393849 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af954652-e499-47cd-992c-c65cf65c55e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:01.394760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af954652-e499-47cd-992c-c65cf65c55e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:01.395213 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af954652-e499-47cd-992c-c65cf65c55e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:01.395548 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c12e3a-5da4-4b13-974f-3de7d778df28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:01.395737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af954652-e499-47cd-992c-c65cf65c55e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:01.396351 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c12e3a-5da4-4b13-974f-3de7d778df28 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:01.396521 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af954652-e499-47cd-992c-c65cf65c55e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:01.396677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c12e3a-5da4-4b13-974f-3de7d778df28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:01.397586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c12e3a-5da4-4b13-974f-3de7d778df28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:01.398722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af954652-e499-47cd-992c-c65cf65c55e9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:01.398912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-65c12e3a-5da4-4b13-974f-3de7d778df28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:01.421176 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 881/1760] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:01.421989 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af954652-e499-47cd-992c-c65cf65c55e9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:01.441274 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af954652-e499-47cd-992c-c65cf65c55e9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:01.446953 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6d95b-da8d-4e6b-8259-a30bd207e2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:01.448031 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6d95b-da8d-4e6b-8259-a30bd207e2af admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:01.448512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6d95b-da8d-4e6b-8259-a30bd207e2af admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:01.449022 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6d95b-da8d-4e6b-8259-a30bd207e2af admin admin] PATH_INFO: `/v3/projects/cb1ef28d7cff479196c6da7f907f0c2d/users/696b0428efa84361be36a6c1efce69d8/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:01.473547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6d95b-da8d-4e6b-8259-a30bd207e2af admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb1ef28d7cff479196c6da7f907f0c2d, user_id=696b0428efa84361be36a6c1efce69d8, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:01.476747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6d95b-da8d-4e6b-8259-a30bd207e2af admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:01.496292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 882/1761] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:01 2022] PUT /identity/v3/projects/cb1ef28d7cff479196c6da7f907f0c2d/users/696b0428efa84361be36a6c1efce69d8/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:01.500692 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6d95b-da8d-4e6b-8259-a30bd207e2af admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:01.518666 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c12e3a-5da4-4b13-974f-3de7d778df28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:01.520132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c12e3a-5da4-4b13-974f-3de7d778df28 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:01.533966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:01.540426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 880/1762] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:01.542723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f603caa-878b-4dd9-97e7-95063f73434c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:01.543417 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f603caa-878b-4dd9-97e7-95063f73434c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:01.544005 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f603caa-878b-4dd9-97e7-95063f73434c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:01.591306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6f603caa-878b-4dd9-97e7-95063f73434c None None] MFA Rules not processed for user `696b0428efa84361be36a6c1efce69d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:01.592863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6f603caa-878b-4dd9-97e7-95063f73434c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:01.662409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6d95b-da8d-4e6b-8259-a30bd207e2af admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:01.677558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f15de7-d09f-4c01-aab7-57392a93a8ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:01.678541 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f15de7-d09f-4c01-aab7-57392a93a8ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:01.678952 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f15de7-d09f-4c01-aab7-57392a93a8ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:01.679358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f15de7-d09f-4c01-aab7-57392a93a8ae admin admin] PATH_INFO: `/v3/projects/be998fc12a3141c58c68248858161ea7/users/6047d21a2f624d3b9c9b1d3852a39848/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:01.711757 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f15de7-d09f-4c01-aab7-57392a93a8ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=be998fc12a3141c58c68248858161ea7, user_id=6047d21a2f624d3b9c9b1d3852a39848, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:01.712273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f15de7-d09f-4c01-aab7-57392a93a8ae admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:01.731824 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 881/1763] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:01 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:01.735248 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f603caa-878b-4dd9-97e7-95063f73434c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:01.737699 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6f603caa-878b-4dd9-97e7-95063f73434c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:01.758418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 883/1764] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:01 2022] PUT /identity/v3/projects/be998fc12a3141c58c68248858161ea7/users/6047d21a2f624d3b9c9b1d3852a39848/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:01.758418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f15de7-d09f-4c01-aab7-57392a93a8ae admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:01.766605 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f603caa-878b-4dd9-97e7-95063f73434c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:01.842130 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f15de7-d09f-4c01-aab7-57392a93a8ae admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:01.847683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1538bfd-fe63-45a9-a0c9-228ed894d0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:01.848310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1538bfd-fe63-45a9-a0c9-228ed894d0a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:01.849181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1538bfd-fe63-45a9-a0c9-228ed894d0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:01.849314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324c477-f495-4040-a1de-af4b02351fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:01.849543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1538bfd-fe63-45a9-a0c9-228ed894d0a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:01.850314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324c477-f495-4040-a1de-af4b02351fd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:01.850445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1538bfd-fe63-45a9-a0c9-228ed894d0a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:01.850630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324c477-f495-4040-a1de-af4b02351fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:01.850966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8324c477-f495-4040-a1de-af4b02351fd9 admin admin] PATH_INFO: `/v3/projects/fbf43b4e2ecb4fa387ad81a4a3a0d7ef` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:01.852039 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324c477-f495-4040-a1de-af4b02351fd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbf43b4e2ecb4fa387ad81a4a3a0d7ef)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:01.852363 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1538bfd-fe63-45a9-a0c9-228ed894d0a0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:01.853992 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8324c477-f495-4040-a1de-af4b02351fd9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:01.866426 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:01.886341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 884/1765] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:01 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:01.887626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35456d8-d3b3-4f7e-999e-ff10057a9081 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:01.887908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35456d8-d3b3-4f7e-999e-ff10057a9081 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:01.888170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35456d8-d3b3-4f7e-999e-ff10057a9081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:01.922679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8324c477-f495-4040-a1de-af4b02351fd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbf43b4e2ecb4fa387ad81a4a3a0d7ef', 'request_id': 'req-8324c477-f495-4040-a1de-af4b02351fd9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:01.930186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e35456d8-d3b3-4f7e-999e-ff10057a9081 None None] MFA Rules not processed for user `6047d21a2f624d3b9c9b1d3852a39848`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:01.931195 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8324c477-f495-4040-a1de-af4b02351fd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbf43b4e2ecb4fa387ad81a4a3a0d7ef', 'request_id': 'req-8324c477-f495-4040-a1de-af4b02351fd9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:01.931666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e35456d8-d3b3-4f7e-999e-ff10057a9081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:01.946017 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8324c477-f495-4040-a1de-af4b02351fd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbf43b4e2ecb4fa387ad81a4a3a0d7ef', 'request_id': 'req-8324c477-f495-4040-a1de-af4b02351fd9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:01.964430 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8324c477-f495-4040-a1de-af4b02351fd9 admin admin] The token cache is being invalidate because project fbf43b4e2ecb4fa387ad81a4a3a0d7ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:01.964430 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8324c477-f495-4040-a1de-af4b02351fd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:01.966651 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8324c477-f495-4040-a1de-af4b02351fd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:01.969385 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 882/1766] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:01 2022] DELETE /identity/v3/projects/fbf43b4e2ecb4fa387ad81a4a3a0d7ef => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:01.970063 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324c477-f495-4040-a1de-af4b02351fd9 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:01.971609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8324c477-f495-4040-a1de-af4b02351fd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:01.992113 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:02.022508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 885/1767] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:01 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:02.024102 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bd213-7b93-4492-8cd8-8fee96784048 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.024396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bd213-7b93-4492-8cd8-8fee96784048 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.024672 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7bd213-7b93-4492-8cd8-8fee96784048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.047859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4b7bd213-7b93-4492-8cd8-8fee96784048 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:02.049096 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4b7bd213-7b93-4492-8cd8-8fee96784048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:02.051309 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8324c477-f495-4040-a1de-af4b02351fd9 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.056102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8fab67-ffb8-441a-81d4-2e69f134a74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.056887 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fab67-ffb8-441a-81d4-2e69f134a74b admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.057047 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 886/1768] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:02.063894 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fab67-ffb8-441a-81d4-2e69f134a74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.063894 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8fab67-ffb8-441a-81d4-2e69f134a74b admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.064110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bd213-7b93-4492-8cd8-8fee96784048 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.064183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fab67-ffb8-441a-81d4-2e69f134a74b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.064257 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4b7bd213-7b93-4492-8cd8-8fee96784048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:02.064326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8fab67-ffb8-441a-81d4-2e69f134a74b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.072781 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7bd213-7b93-4492-8cd8-8fee96784048 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.079099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbb62e-b479-463f-8ae5-c246160c950a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.079727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbb62e-b479-463f-8ae5-c246160c950a admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.080000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbb62e-b479-463f-8ae5-c246160c950a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.080274 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcbb62e-b479-463f-8ae5-c246160c950a admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.080896 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbb62e-b479-463f-8ae5-c246160c950a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.083213 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcbb62e-b479-463f-8ae5-c246160c950a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.083355 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 883/1769] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:01 2022] POST /identity/v3/users => generated 298 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:02.084407 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8de8c-c766-49ac-a2b0-3c4fc553e46c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.084693 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8de8c-c766-49ac-a2b0-3c4fc553e46c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.084962 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8de8c-c766-49ac-a2b0-3c4fc553e46c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.106099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-30d8de8c-c766-49ac-a2b0-3c4fc553e46c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:02.107323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-30d8de8c-c766-49ac-a2b0-3c4fc553e46c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:02.114276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 887/1770] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:02 2022] POST /identity/v3/projects => generated 399 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:02.115128 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbb62e-b479-463f-8ae5-c246160c950a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.119638 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 884/1771] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:02.120560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d8de8c-c766-49ac-a2b0-3c4fc553e46c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.131022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d8de8c-c766-49ac-a2b0-3c4fc553e46c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.189846 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcbb62e-b479-463f-8ae5-c246160c950a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.195909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6aa21-ed01-4de4-805f-00f84aaab992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.196525 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6aa21-ed01-4de4-805f-00f84aaab992 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.196806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6aa21-ed01-4de4-805f-00f84aaab992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.197076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6aa21-ed01-4de4-805f-00f84aaab992 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.197721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6aa21-ed01-4de4-805f-00f84aaab992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.199359 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6aa21-ed01-4de4-805f-00f84aaab992 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.203742 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1e24d-ae5b-4c75-bb3c-7bffcc08bc0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.205355 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1e24d-ae5b-4c75-bb3c-7bffcc08bc0c admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.205722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1e24d-ae5b-4c75-bb3c-7bffcc08bc0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.206091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb1e24d-ae5b-4c75-bb3c-7bffcc08bc0c admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.206880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1e24d-ae5b-4c75-bb3c-7bffcc08bc0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.208866 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb1e24d-ae5b-4c75-bb3c-7bffcc08bc0c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.213459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 888/1772] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:02.213459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6aa21-ed01-4de4-805f-00f84aaab992 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.220892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6aa21-ed01-4de4-805f-00f84aaab992 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.226766 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00140f5b-175d-4d6d-bb65-3bf9e0301c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.226766 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00140f5b-175d-4d6d-bb65-3bf9e0301c03 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.227597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00140f5b-175d-4d6d-bb65-3bf9e0301c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.227597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00140f5b-175d-4d6d-bb65-3bf9e0301c03 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.227901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00140f5b-175d-4d6d-bb65-3bf9e0301c03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:02.229152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00140f5b-175d-4d6d-bb65-3bf9e0301c03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.229851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00140f5b-175d-4d6d-bb65-3bf9e0301c03 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.236054 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 885/1773] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:02 2022] POST /identity/v3/users => generated 306 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:02.236877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1e24d-ae5b-4c75-bb3c-7bffcc08bc0c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.243589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 889/1774] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:02.244370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00140f5b-175d-4d6d-bb65-3bf9e0301c03 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.247610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb1e24d-ae5b-4c75-bb3c-7bffcc08bc0c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.252445 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1e0dd-d02b-490b-9d34-8809c14ac0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.253120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1e0dd-d02b-490b-9d34-8809c14ac0f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.253384 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1e0dd-d02b-490b-9d34-8809c14ac0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.253655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1e0dd-d02b-490b-9d34-8809c14ac0f3 admin admin] PATH_INFO: `/v3/projects/fd026be13d63446ebefbdb226e76e022/users/0596914323ee4d7a8d3d67c40c0af2e3/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.254265 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00140f5b-175d-4d6d-bb65-3bf9e0301c03 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.259130 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c39246-ba97-4716-813d-8a0e2e0bcb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.259733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c39246-ba97-4716-813d-8a0e2e0bcb9a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.260011 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c39246-ba97-4716-813d-8a0e2e0bcb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.260279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c39246-ba97-4716-813d-8a0e2e0bcb9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.260917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c39246-ba97-4716-813d-8a0e2e0bcb9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.262524 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c39246-ba97-4716-813d-8a0e2e0bcb9a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.272659 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 890/1775] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:02.273457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c39246-ba97-4716-813d-8a0e2e0bcb9a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.276924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1e0dd-d02b-490b-9d34-8809c14ac0f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd026be13d63446ebefbdb226e76e022, user_id=0596914323ee4d7a8d3d67c40c0af2e3, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.282174 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1e0dd-d02b-490b-9d34-8809c14ac0f3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.286782 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c39246-ba97-4716-813d-8a0e2e0bcb9a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.291618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb7b84-abc7-4483-8b6c-1bda2a03221f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.292221 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb7b84-abc7-4483-8b6c-1bda2a03221f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.292497 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb7b84-abc7-4483-8b6c-1bda2a03221f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.292767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cb7b84-abc7-4483-8b6c-1bda2a03221f admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.293446 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb7b84-abc7-4483-8b6c-1bda2a03221f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.295360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cb7b84-abc7-4483-8b6c-1bda2a03221f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.296511 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 886/1776] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:02 2022] PUT /identity/v3/projects/fd026be13d63446ebefbdb226e76e022/users/0596914323ee4d7a8d3d67c40c0af2e3/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:02.297316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1e0dd-d02b-490b-9d34-8809c14ac0f3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.329308 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 891/1777] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:02 2022] POST /identity/v3/projects => generated 403 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:02.330322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb7b84-abc7-4483-8b6c-1bda2a03221f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.335780 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:02.368859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1e0dd-d02b-490b-9d34-8809c14ac0f3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.378600 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:02.404864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cb7b84-abc7-4483-8b6c-1bda2a03221f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.410617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e834bb-3bc1-4579-afbf-22211938eb55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.411165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e834bb-3bc1-4579-afbf-22211938eb55 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.411442 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e834bb-3bc1-4579-afbf-22211938eb55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.411713 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e834bb-3bc1-4579-afbf-22211938eb55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.412362 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e834bb-3bc1-4579-afbf-22211938eb55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.413966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e834bb-3bc1-4579-afbf-22211938eb55 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.425113 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 892/1778] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:02.425997 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e834bb-3bc1-4579-afbf-22211938eb55 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.436295 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e834bb-3bc1-4579-afbf-22211938eb55 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.438874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832fe60-01d4-47c8-a582-26e39651c344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.439703 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832fe60-01d4-47c8-a582-26e39651c344 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.440057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832fe60-01d4-47c8-a582-26e39651c344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.440412 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6832fe60-01d4-47c8-a582-26e39651c344 admin admin] PATH_INFO: `/v3/projects/c22b4103e8b540e39ef54e606c18f6ab/users/ef34cfe262b44f1a8b307e06d33db2e1/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.441299 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d49bc-bf23-4e43-8c87-4e1fbbd50744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.441854 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d49bc-bf23-4e43-8c87-4e1fbbd50744 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.442134 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d49bc-bf23-4e43-8c87-4e1fbbd50744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.442446 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883d49bc-bf23-4e43-8c87-4e1fbbd50744 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.443075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d49bc-bf23-4e43-8c87-4e1fbbd50744 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.445000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883d49bc-bf23-4e43-8c87-4e1fbbd50744 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.459659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6832fe60-01d4-47c8-a582-26e39651c344 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c22b4103e8b540e39ef54e606c18f6ab, user_id=ef34cfe262b44f1a8b307e06d33db2e1, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.462392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6832fe60-01d4-47c8-a582-26e39651c344 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.472589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 893/1779] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:02 2022] POST /identity/v3/users => generated 302 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:02.473465 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d49bc-bf23-4e43-8c87-4e1fbbd50744 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.479943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 887/1780] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:02 2022] PUT /identity/v3/projects/c22b4103e8b540e39ef54e606c18f6ab/users/ef34cfe262b44f1a8b307e06d33db2e1/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:02.481116 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832fe60-01d4-47c8-a582-26e39651c344 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.488602 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883d49bc-bf23-4e43-8c87-4e1fbbd50744 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.564924 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881cb2c-8218-402f-88d4-d0ba38fb856a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.565677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881cb2c-8218-402f-88d4-d0ba38fb856a admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.566076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881cb2c-8218-402f-88d4-d0ba38fb856a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.566370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9881cb2c-8218-402f-88d4-d0ba38fb856a admin admin] PATH_INFO: `/v3/projects/fd026be13d63446ebefbdb226e76e022/users/0596914323ee4d7a8d3d67c40c0af2e3/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.566513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6832fe60-01d4-47c8-a582-26e39651c344 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.568529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881cb2c-8218-402f-88d4-d0ba38fb856a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd026be13d63446ebefbdb226e76e022, user_id=0596914323ee4d7a8d3d67c40c0af2e3, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.571516 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9881cb2c-8218-402f-88d4-d0ba38fb856a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.572786 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94832650-5c95-4579-b8fa-c9123438826b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.573566 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94832650-5c95-4579-b8fa-c9123438826b admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.573880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94832650-5c95-4579-b8fa-c9123438826b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.574349 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94832650-5c95-4579-b8fa-c9123438826b admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.575008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94832650-5c95-4579-b8fa-c9123438826b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.576696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94832650-5c95-4579-b8fa-c9123438826b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.585585 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 894/1781] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:02 2022] PUT /identity/v3/projects/fd026be13d63446ebefbdb226e76e022/users/0596914323ee4d7a8d3d67c40c0af2e3/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:02.586728 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881cb2c-8218-402f-88d4-d0ba38fb856a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.590515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 888/1782] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:02.593788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94832650-5c95-4579-b8fa-c9123438826b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.604278 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94832650-5c95-4579-b8fa-c9123438826b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.664608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9881cb2c-8218-402f-88d4-d0ba38fb856a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.667673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b58c40-f73c-49e3-a9bf-03a9781f4777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.668337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b58c40-f73c-49e3-a9bf-03a9781f4777 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.668617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b58c40-f73c-49e3-a9bf-03a9781f4777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.668885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b58c40-f73c-49e3-a9bf-03a9781f4777 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.669523 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b58c40-f73c-49e3-a9bf-03a9781f4777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.671916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b58c40-f73c-49e3-a9bf-03a9781f4777 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.672941 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62236aba-622e-4f09-9da8-d09a0cfd4745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.673718 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62236aba-622e-4f09-9da8-d09a0cfd4745 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.673985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62236aba-622e-4f09-9da8-d09a0cfd4745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.674373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62236aba-622e-4f09-9da8-d09a0cfd4745 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.675471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62236aba-622e-4f09-9da8-d09a0cfd4745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.677028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62236aba-622e-4f09-9da8-d09a0cfd4745 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.684348 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 889/1783] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:02.685815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc07f1d-83c8-42e4-89bf-8205b8f584c7 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.686194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc07f1d-83c8-42e4-89bf-8205b8f584c7 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.686627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc07f1d-83c8-42e4-89bf-8205b8f584c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.689695 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 895/1784] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:02.690813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62236aba-622e-4f09-9da8-d09a0cfd4745 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.705933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62236aba-622e-4f09-9da8-d09a0cfd4745 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.711126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb3691-565e-4391-94d4-a5676fb9dc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.711797 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb3691-565e-4391-94d4-a5676fb9dc19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.712670 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb3691-565e-4391-94d4-a5676fb9dc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.712950 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eb3691-565e-4391-94d4-a5676fb9dc19 admin admin] PATH_INFO: `/v3/projects/4c97cc48060d4a4c99d8cd566250f972/users/d848a95ccb434723b29fd47b57519b10/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.725413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:02.738630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2bc07f1d-83c8-42e4-89bf-8205b8f584c7 None None] MFA Rules not processed for user `0596914323ee4d7a8d3d67c40c0af2e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:02.741817 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2bc07f1d-83c8-42e4-89bf-8205b8f584c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:02.745203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb3691-565e-4391-94d4-a5676fb9dc19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c97cc48060d4a4c99d8cd566250f972, user_id=d848a95ccb434723b29fd47b57519b10, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.749464 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eb3691-565e-4391-94d4-a5676fb9dc19 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.762338 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:02.774368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 896/1785] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:02 2022] PUT /identity/v3/projects/4c97cc48060d4a4c99d8cd566250f972/users/d848a95ccb434723b29fd47b57519b10/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:02.775733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb3691-565e-4391-94d4-a5676fb9dc19 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.841579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 890/1786] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:02 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:02.844018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d23eab-2942-4184-a098-b178a600ed3c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.844486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d23eab-2942-4184-a098-b178a600ed3c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.845052 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d23eab-2942-4184-a098-b178a600ed3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.871885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-90d23eab-2942-4184-a098-b178a600ed3c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:02.873321 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-90d23eab-2942-4184-a098-b178a600ed3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:02.921196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eb3691-565e-4391-94d4-a5676fb9dc19 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:02.934293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa110a35-60c2-4e66-b226-785842da687a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:02.935325 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa110a35-60c2-4e66-b226-785842da687a admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:02.935701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa110a35-60c2-4e66-b226-785842da687a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:02.936074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa110a35-60c2-4e66-b226-785842da687a admin admin] PATH_INFO: `/v3/projects/c22b4103e8b540e39ef54e606c18f6ab/users/ef34cfe262b44f1a8b307e06d33db2e1/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:02.939772 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa110a35-60c2-4e66-b226-785842da687a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c22b4103e8b540e39ef54e606c18f6ab, user_id=ef34cfe262b44f1a8b307e06d33db2e1, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:02.945038 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa110a35-60c2-4e66-b226-785842da687a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:02.967961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 891/1787] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:02.978730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 897/1788] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:02 2022] PUT /identity/v3/projects/c22b4103e8b540e39ef54e606c18f6ab/users/ef34cfe262b44f1a8b307e06d33db2e1/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:02.979749 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa110a35-60c2-4e66-b226-785842da687a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.981953 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aa110a35-60c2-4e66-b226-785842da687a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:02.984361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d23eab-2942-4184-a098-b178a600ed3c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:02.986423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-90d23eab-2942-4184-a098-b178a600ed3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:03.074453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d23eab-2942-4184-a098-b178a600ed3c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:03.101318 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa110a35-60c2-4e66-b226-785842da687a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:03.110828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c054232-e133-4a40-ab4f-cea583150435 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:03.112954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c054232-e133-4a40-ab4f-cea583150435 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:03.113495 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c054232-e133-4a40-ab4f-cea583150435 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:03.113963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c054232-e133-4a40-ab4f-cea583150435 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:03.203675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed390036-680f-467e-95c5-cefe44076255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:03.204310 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed390036-680f-467e-95c5-cefe44076255 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:03.204592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed390036-680f-467e-95c5-cefe44076255 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:03.204874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed390036-680f-467e-95c5-cefe44076255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:03.226304 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c054232-e133-4a40-ab4f-cea583150435 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:03.228516 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c054232-e133-4a40-ab4f-cea583150435 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:03.240667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:03.247330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 898/1789] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:03.258068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d333a1-95c0-471d-99d7-12e7241e8b46 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:03.259720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d333a1-95c0-471d-99d7-12e7241e8b46 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:03.260009 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d333a1-95c0-471d-99d7-12e7241e8b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:03.318278 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b6d333a1-95c0-471d-99d7-12e7241e8b46 None None] MFA Rules not processed for user `ef34cfe262b44f1a8b307e06d33db2e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:03.319965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b6d333a1-95c0-471d-99d7-12e7241e8b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:03.335268 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed390036-680f-467e-95c5-cefe44076255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:03.340846 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed390036-680f-467e-95c5-cefe44076255 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:03.349848 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:03.366745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 892/1790] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:03.518213 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 899/1791] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:03 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:03.583874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed390036-680f-467e-95c5-cefe44076255 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:03.627166 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed390036-680f-467e-95c5-cefe44076255 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:03.627580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d333a1-95c0-471d-99d7-12e7241e8b46 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:03.630170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b6d333a1-95c0-471d-99d7-12e7241e8b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:03.647167 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a01c2-cf9a-474a-93e1-d7f443b0b742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:03.648087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a01c2-cf9a-474a-93e1-d7f443b0b742 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:03.649059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a01c2-cf9a-474a-93e1-d7f443b0b742 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:03.649344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a01c2-cf9a-474a-93e1-d7f443b0b742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:03.650340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d333a1-95c0-471d-99d7-12e7241e8b46 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:03.668370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6a01c2-cf9a-474a-93e1-d7f443b0b742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:03.671125 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd0200-c76f-486e-9f7d-500ea668f5fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:03.671541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b6a01c2-cf9a-474a-93e1-d7f443b0b742 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:03.671783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd0200-c76f-486e-9f7d-500ea668f5fe service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:03.672074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd0200-c76f-486e-9f7d-500ea668f5fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:03.672357 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd0200-c76f-486e-9f7d-500ea668f5fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:03.678618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-86fd0200-c76f-486e-9f7d-500ea668f5fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:03.688943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 893/1792] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:03 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:03.828891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd0200-c76f-486e-9f7d-500ea668f5fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:03.831202 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fd0200-c76f-486e-9f7d-500ea668f5fe service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:03.844699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 900/1793] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:03.976401 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a01c2-cf9a-474a-93e1-d7f443b0b742 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:03.988883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a01c2-cf9a-474a-93e1-d7f443b0b742 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:04.000457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52137e8c-45aa-4296-9fec-254641b7c01d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:04.001344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52137e8c-45aa-4296-9fec-254641b7c01d service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:04.001869 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52137e8c-45aa-4296-9fec-254641b7c01d service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:04.002365 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52137e8c-45aa-4296-9fec-254641b7c01d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:04.127083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52137e8c-45aa-4296-9fec-254641b7c01d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:04.128963 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52137e8c-45aa-4296-9fec-254641b7c01d service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:04.142470 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 894/1794] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:03 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:04.443949 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0c4fa-33ab-4d9d-846c-920cc15b7e6c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:04.445021 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0c4fa-33ab-4d9d-846c-920cc15b7e6c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:04.445704 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d0c4fa-33ab-4d9d-846c-920cc15b7e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:04.489303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52137e8c-45aa-4296-9fec-254641b7c01d service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:04.491885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-52137e8c-45aa-4296-9fec-254641b7c01d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:04.517869 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52137e8c-45aa-4296-9fec-254641b7c01d service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:04.523422 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-43d0c4fa-33ab-4d9d-846c-920cc15b7e6c None None] MFA Rules not processed for user `d848a95ccb434723b29fd47b57519b10`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:04.524857 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-43d0c4fa-33ab-4d9d-846c-920cc15b7e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:04.529287 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e42f6c7-1362-41f2-9250-a8505a3c4621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:04.531377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42f6c7-1362-41f2-9250-a8505a3c4621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:04.531628 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42f6c7-1362-41f2-9250-a8505a3c4621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:04.534063 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e42f6c7-1362-41f2-9250-a8505a3c4621 admin admin] PATH_INFO: `/v3/users/bb79ba1175694ad5928444f66064a1e2` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:04.535690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42f6c7-1362-41f2-9250-a8505a3c4621 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb79ba1175694ad5928444f66064a1e2)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:04.543345 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e42f6c7-1362-41f2-9250-a8505a3c4621 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:04.650211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2e42f6c7-1362-41f2-9250-a8505a3c4621 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb79ba1175694ad5928444f66064a1e2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:04.683205 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2e42f6c7-1362-41f2-9250-a8505a3c4621 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb79ba1175694ad5928444f66064a1e2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:04.724404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2e42f6c7-1362-41f2-9250-a8505a3c4621 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb79ba1175694ad5928444f66064a1e2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:04.725612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2e42f6c7-1362-41f2-9250-a8505a3c4621 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb79ba1175694ad5928444f66064a1e2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:04.743856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2e42f6c7-1362-41f2-9250-a8505a3c4621 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb79ba1175694ad5928444f66064a1e2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:04.760127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 901/1795] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:04 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:04.763031 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0c4fa-33ab-4d9d-846c-920cc15b7e6c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:04.763507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 895/1796] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:04 2022] DELETE /identity/v3/users/bb79ba1175694ad5928444f66064a1e2 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:04.768224 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb848d7-a754-49ec-b731-8716795df0f1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:04.768622 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb848d7-a754-49ec-b731-8716795df0f1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:04.768791 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb848d7-a754-49ec-b731-8716795df0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:04.777752 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d0c4fa-33ab-4d9d-846c-920cc15b7e6c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:04.812650 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ceb848d7-a754-49ec-b731-8716795df0f1 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:04.823722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ceb848d7-a754-49ec-b731-8716795df0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:04.848592 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:04.860741 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:04.870898 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed1800-1ec3-4f12-8eb8-aaad07beb154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:04.871289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed1800-1ec3-4f12-8eb8-aaad07beb154 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:04.872287 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed1800-1ec3-4f12-8eb8-aaad07beb154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:04.872573 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed1800-1ec3-4f12-8eb8-aaad07beb154 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:04.873358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed1800-1ec3-4f12-8eb8-aaad07beb154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:04.875860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed1800-1ec3-4f12-8eb8-aaad07beb154 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:04.933906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 902/1797] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:04 2022] POST /identity/v3/projects => generated 405 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:04.935172 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed1800-1ec3-4f12-8eb8-aaad07beb154 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:04.939776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c0ed1800-1ec3-4f12-8eb8-aaad07beb154 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:04.960525 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 896/1798] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:04.972659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb848d7-a754-49ec-b731-8716795df0f1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:04.974084 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ceb848d7-a754-49ec-b731-8716795df0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:05.040774 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed1800-1ec3-4f12-8eb8-aaad07beb154 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:05.045796 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060c999-a58a-4d69-b083-40a600981df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:05.046370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb848d7-a754-49ec-b731-8716795df0f1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:05.046542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060c999-a58a-4d69-b083-40a600981df8 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:05.046937 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060c999-a58a-4d69-b083-40a600981df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:05.047279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8060c999-a58a-4d69-b083-40a600981df8 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:05.047999 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060c999-a58a-4d69-b083-40a600981df8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:05.050084 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8060c999-a58a-4d69-b083-40a600981df8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:05.080560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 903/1799] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:04 2022] POST /identity/v3/users => generated 309 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:05.082132 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060c999-a58a-4d69-b083-40a600981df8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:05.095873 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8060c999-a58a-4d69-b083-40a600981df8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:05.102639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c91c09-6c1e-47b0-b57a-d7893797aad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:05.103623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c91c09-6c1e-47b0-b57a-d7893797aad1 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:05.104025 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c91c09-6c1e-47b0-b57a-d7893797aad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:05.104400 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c91c09-6c1e-47b0-b57a-d7893797aad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:05.105396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c91c09-6c1e-47b0-b57a-d7893797aad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:05.110384 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d521c3-2583-469c-b240-a2e537a49e84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:05.111002 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d521c3-2583-469c-b240-a2e537a49e84 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:05.111285 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d521c3-2583-469c-b240-a2e537a49e84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:05.111559 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c91c09-6c1e-47b0-b57a-d7893797aad1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:05.111673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d521c3-2583-469c-b240-a2e537a49e84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:05.124728 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d521c3-2583-469c-b240-a2e537a49e84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:05.124876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 904/1800] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:05.125970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d521c3-2583-469c-b240-a2e537a49e84 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:05.129380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c91c09-6c1e-47b0-b57a-d7893797aad1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:05.138053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 897/1801] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:05.142898 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c91c09-6c1e-47b0-b57a-d7893797aad1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:05.149801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d28a8-2a47-4baa-8daf-0c38890ed442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:05.150693 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d28a8-2a47-4baa-8daf-0c38890ed442 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:05.150973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d28a8-2a47-4baa-8daf-0c38890ed442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:05.151248 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d28a8-2a47-4baa-8daf-0c38890ed442 admin admin] PATH_INFO: `/v3/projects/f2667661fd1f4e0bb91b453d32236915/users/5b1b7484ec094b6e8422d04fce4d878e/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:05.173438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d28a8-2a47-4baa-8daf-0c38890ed442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2667661fd1f4e0bb91b453d32236915, user_id=5b1b7484ec094b6e8422d04fce4d878e, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:05.175631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d28a8-2a47-4baa-8daf-0c38890ed442 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:05.191992 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 905/1802] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:05 2022] PUT /identity/v3/projects/f2667661fd1f4e0bb91b453d32236915/users/5b1b7484ec094b6e8422d04fce4d878e/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:05.195951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d521c3-2583-469c-b240-a2e537a49e84 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:05.207314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d521c3-2583-469c-b240-a2e537a49e84 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:05.296528 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486714a-90a8-4edf-bdac-f67974339f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:05.299935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486714a-90a8-4edf-bdac-f67974339f5d admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:05.300368 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486714a-90a8-4edf-bdac-f67974339f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:05.300661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0486714a-90a8-4edf-bdac-f67974339f5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:05.301357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486714a-90a8-4edf-bdac-f67974339f5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:05.303305 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0486714a-90a8-4edf-bdac-f67974339f5d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:05.314844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 898/1803] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:05 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:05.319140 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d28a8-2a47-4baa-8daf-0c38890ed442 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:05.333077 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d28a8-2a47-4baa-8daf-0c38890ed442 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:05.338689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb208d4-642f-461b-bb78-83fa398459b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:05.339373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb208d4-642f-461b-bb78-83fa398459b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:05.339855 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb208d4-642f-461b-bb78-83fa398459b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:05.340166 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb208d4-642f-461b-bb78-83fa398459b3 admin admin] PATH_INFO: `/v3/projects/f2667661fd1f4e0bb91b453d32236915/users/5b1b7484ec094b6e8422d04fce4d878e/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:05.343562 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb208d4-642f-461b-bb78-83fa398459b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2667661fd1f4e0bb91b453d32236915, user_id=5b1b7484ec094b6e8422d04fce4d878e, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:05.347008 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb208d4-642f-461b-bb78-83fa398459b3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:05.359688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 906/1804] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:05 2022] PUT /identity/v3/projects/f2667661fd1f4e0bb91b453d32236915/users/5b1b7484ec094b6e8422d04fce4d878e/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:05.512537 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486714a-90a8-4edf-bdac-f67974339f5d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:05.517880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0486714a-90a8-4edf-bdac-f67974339f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:05.557658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0486714a-90a8-4edf-bdac-f67974339f5d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:05.575745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:05.648508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc31e74b-87ff-4495-8e54-759438122233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:05.649868 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31e74b-87ff-4495-8e54-759438122233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:05.650181 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31e74b-87ff-4495-8e54-759438122233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:05.650480 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc31e74b-87ff-4495-8e54-759438122233 admin admin] PATH_INFO: `/v3/projects/e1d7efa8baf2432886619cfe25aa706d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:05.651966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31e74b-87ff-4495-8e54-759438122233 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1d7efa8baf2432886619cfe25aa706d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:05.654289 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc31e74b-87ff-4495-8e54-759438122233 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:05.723983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dc31e74b-87ff-4495-8e54-759438122233 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1d7efa8baf2432886619cfe25aa706d', 'request_id': 'req-dc31e74b-87ff-4495-8e54-759438122233'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:05.733685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dc31e74b-87ff-4495-8e54-759438122233 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1d7efa8baf2432886619cfe25aa706d', 'request_id': 'req-dc31e74b-87ff-4495-8e54-759438122233'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:05.744222 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dc31e74b-87ff-4495-8e54-759438122233 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1d7efa8baf2432886619cfe25aa706d', 'request_id': 'req-dc31e74b-87ff-4495-8e54-759438122233'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:05.763952 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dc31e74b-87ff-4495-8e54-759438122233 admin admin] The token cache is being invalidate because project e1d7efa8baf2432886619cfe25aa706d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:05.764980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dc31e74b-87ff-4495-8e54-759438122233 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:05.765690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dc31e74b-87ff-4495-8e54-759438122233 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:05.768117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 899/1805] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:05 2022] DELETE /identity/v3/projects/e1d7efa8baf2432886619cfe25aa706d => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:05.772913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632eed8-a3a1-4348-92cc-6f78159bf858 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:05.773383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632eed8-a3a1-4348-92cc-6f78159bf858 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:05.773458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632eed8-a3a1-4348-92cc-6f78159bf858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:05.801397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4632eed8-a3a1-4348-92cc-6f78159bf858 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:05.802811 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4632eed8-a3a1-4348-92cc-6f78159bf858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:05.917621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 907/1806] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:05.924952 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ad5fe-3198-448c-8c0b-a3942105be7f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:05.925241 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ad5fe-3198-448c-8c0b-a3942105be7f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:05.925517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1ad5fe-3198-448c-8c0b-a3942105be7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:05.952264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3e1ad5fe-3198-448c-8c0b-a3942105be7f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:05.953561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3e1ad5fe-3198-448c-8c0b-a3942105be7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:05.975801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 900/1807] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:05.978927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632eed8-a3a1-4348-92cc-6f78159bf858 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:05.991922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632eed8-a3a1-4348-92cc-6f78159bf858 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:06.000156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f349f7-b1af-4824-a449-af1b814c29f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:06.000781 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f349f7-b1af-4824-a449-af1b814c29f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:06.001064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f349f7-b1af-4824-a449-af1b814c29f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:06.001368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f349f7-b1af-4824-a449-af1b814c29f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:06.002000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f349f7-b1af-4824-a449-af1b814c29f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:06.002356 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f349f7-b1af-4824-a449-af1b814c29f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:06.003984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f349f7-b1af-4824-a449-af1b814c29f0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:06.017444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:06.025481 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 908/1808] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:06.026624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ad5fe-3198-448c-8c0b-a3942105be7f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:06.044857 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1ad5fe-3198-448c-8c0b-a3942105be7f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:06.060450 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4a3a0-3ccc-44be-bf1b-97ba6fccbd48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:06.062404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4a3a0-3ccc-44be-bf1b-97ba6fccbd48 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:06.062585 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4a3a0-3ccc-44be-bf1b-97ba6fccbd48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:06.062866 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb4a3a0-3ccc-44be-bf1b-97ba6fccbd48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:06.063719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4a3a0-3ccc-44be-bf1b-97ba6fccbd48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:06.067056 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb4a3a0-3ccc-44be-bf1b-97ba6fccbd48 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:06.131950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 901/1809] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:06 2022] POST /identity/v3/projects => generated 363 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:06.136176 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f349f7-b1af-4824-a449-af1b814c29f0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:06.151341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f349f7-b1af-4824-a449-af1b814c29f0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:06.233861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10cd27-8497-40e3-9113-281597576a4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:06.234993 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10cd27-8497-40e3-9113-281597576a4d admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:06.235454 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10cd27-8497-40e3-9113-281597576a4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:06.235938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e10cd27-8497-40e3-9113-281597576a4d admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:06.236810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10cd27-8497-40e3-9113-281597576a4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:06.239481 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e10cd27-8497-40e3-9113-281597576a4d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:06.272096 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 909/1810] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:06 2022] POST /identity/v3/users => generated 282 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:06.273326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4a3a0-3ccc-44be-bf1b-97ba6fccbd48 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:06.299109 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb4a3a0-3ccc-44be-bf1b-97ba6fccbd48 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:06.304903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e919e-149f-4473-b29c-7dff0454f258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:06.305521 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e919e-149f-4473-b29c-7dff0454f258 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:06.305794 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e919e-149f-4473-b29c-7dff0454f258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:06.306069 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5e919e-149f-4473-b29c-7dff0454f258 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:06.306804 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e919e-149f-4473-b29c-7dff0454f258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:06.308852 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5e919e-149f-4473-b29c-7dff0454f258 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:06.334269 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 902/1811] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:06.338324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10cd27-8497-40e3-9113-281597576a4d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:06.351303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e10cd27-8497-40e3-9113-281597576a4d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:06.356753 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c61ead-ad77-46c1-be7b-91fb8095a2da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:06.357379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c61ead-ad77-46c1-be7b-91fb8095a2da admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:06.357659 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c61ead-ad77-46c1-be7b-91fb8095a2da admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:06.357932 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c61ead-ad77-46c1-be7b-91fb8095a2da admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:06.358836 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c61ead-ad77-46c1-be7b-91fb8095a2da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:06.360650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c61ead-ad77-46c1-be7b-91fb8095a2da admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:06.374860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 910/1812] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:06.380062 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e919e-149f-4473-b29c-7dff0454f258 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:06.394091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5e919e-149f-4473-b29c-7dff0454f258 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:06.399816 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7595983e-0127-426b-8d40-f12a78b365d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:06.401472 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595983e-0127-426b-8d40-f12a78b365d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:06.401472 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595983e-0127-426b-8d40-f12a78b365d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:06.401472 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7595983e-0127-426b-8d40-f12a78b365d4 admin admin] PATH_INFO: `/v3/projects/8f3d9ae76f4b408da4ab498d3405f36b/users/b40f119e1cc44bffb9e3554ea473a417/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:06.433426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595983e-0127-426b-8d40-f12a78b365d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f3d9ae76f4b408da4ab498d3405f36b, user_id=b40f119e1cc44bffb9e3554ea473a417, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:06.436876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7595983e-0127-426b-8d40-f12a78b365d4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:06.452948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb19a9-d437-4aaa-a5fc-e53c18e52590 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:06.453533 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb19a9-d437-4aaa-a5fc-e53c18e52590 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:06.454174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfb19a9-d437-4aaa-a5fc-e53c18e52590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:06.462033 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 903/1813] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:06 2022] PUT /identity/v3/projects/8f3d9ae76f4b408da4ab498d3405f36b/users/b40f119e1cc44bffb9e3554ea473a417/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:06.474036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57c55d-554e-4acf-9286-e68db06bfc63 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:06.474498 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57c55d-554e-4acf-9286-e68db06bfc63 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:06.474873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b57c55d-554e-4acf-9286-e68db06bfc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:06.489976 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1dfb19a9-d437-4aaa-a5fc-e53c18e52590 None None] MFA Rules not processed for user `5b1b7484ec094b6e8422d04fce4d878e`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:06.491343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1dfb19a9-d437-4aaa-a5fc-e53c18e52590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:06.516984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2b57c55d-554e-4acf-9286-e68db06bfc63 None None] MFA Rules not processed for user `b40f119e1cc44bffb9e3554ea473a417`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:06.518204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2b57c55d-554e-4acf-9286-e68db06bfc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:06.560597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:06.582043 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 911/1814] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:06 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:06.610446 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 904/1815] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:06.616522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb19a9-d437-4aaa-a5fc-e53c18e52590 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:06.692430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfb19a9-d437-4aaa-a5fc-e53c18e52590 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:06.697767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c2351-6a40-4e70-97a8-c1244a1c7bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:06.698489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c2351-6a40-4e70-97a8-c1244a1c7bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:06.698844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c2351-6a40-4e70-97a8-c1244a1c7bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:06.699127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c2351-6a40-4e70-97a8-c1244a1c7bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:06.699788 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c2351-6a40-4e70-97a8-c1244a1c7bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:06.701848 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c2351-6a40-4e70-97a8-c1244a1c7bd5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:06.707825 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:06.745999 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 912/1816] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:06 2022] POST /identity/v3/projects => generated 363 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:06.749952 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57c55d-554e-4acf-9286-e68db06bfc63 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:06.762328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b57c55d-554e-4acf-9286-e68db06bfc63 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:06.862341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc5f56-b8c8-407e-b779-c7090e420eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:06.863328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc5f56-b8c8-407e-b779-c7090e420eb1 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:06.864258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc5f56-b8c8-407e-b779-c7090e420eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:06.864595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc5f56-b8c8-407e-b779-c7090e420eb1 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:06.865254 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc5f56-b8c8-407e-b779-c7090e420eb1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:06.868146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc5f56-b8c8-407e-b779-c7090e420eb1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:06.898154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 905/1817] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:06 2022] POST /identity/v3/users => generated 288 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:06.899343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c2351-6a40-4e70-97a8-c1244a1c7bd5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:06.923168 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c2351-6a40-4e70-97a8-c1244a1c7bd5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:06.923168 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc9da4-ab33-440a-b4f3-3ce2e0bddfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:06.923168 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc9da4-ab33-440a-b4f3-3ce2e0bddfb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:06.923168 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc9da4-ab33-440a-b4f3-3ce2e0bddfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:06.923447 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc9da4-ab33-440a-b4f3-3ce2e0bddfb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:06.923447 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc9da4-ab33-440a-b4f3-3ce2e0bddfb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:06.923447 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc9da4-ab33-440a-b4f3-3ce2e0bddfb9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:06.943915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 913/1818] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:06.948957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc5f56-b8c8-407e-b779-c7090e420eb1 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:06.978418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc5f56-b8c8-407e-b779-c7090e420eb1 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:06.978418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82838b4a-c938-4d1f-a551-b7a85737fca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:06.978418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82838b4a-c938-4d1f-a551-b7a85737fca3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:06.978418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82838b4a-c938-4d1f-a551-b7a85737fca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:06.978716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82838b4a-c938-4d1f-a551-b7a85737fca3 admin admin] PATH_INFO: `/v3/projects/379c68903ed44ed5aa003f6e2616934e/users/c67d2a012577468a875730c74b174da0/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:06.988699 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82838b4a-c938-4d1f-a551-b7a85737fca3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=379c68903ed44ed5aa003f6e2616934e, user_id=c67d2a012577468a875730c74b174da0, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:06.992335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82838b4a-c938-4d1f-a551-b7a85737fca3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:07.016246 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 906/1819] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:06 2022] PUT /identity/v3/projects/379c68903ed44ed5aa003f6e2616934e/users/c67d2a012577468a875730c74b174da0/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:07.017750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc9da4-ab33-440a-b4f3-3ce2e0bddfb9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:07.030831 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc9da4-ab33-440a-b4f3-3ce2e0bddfb9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:07.106551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1837c2-ff10-44b0-8e19-dee4a490a879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:07.107270 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1837c2-ff10-44b0-8e19-dee4a490a879 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:07.107640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1837c2-ff10-44b0-8e19-dee4a490a879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:07.108013 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1837c2-ff10-44b0-8e19-dee4a490a879 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:07.109751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1837c2-ff10-44b0-8e19-dee4a490a879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:07.111907 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1837c2-ff10-44b0-8e19-dee4a490a879 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:07.115135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82838b4a-c938-4d1f-a551-b7a85737fca3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:07.117164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-82838b4a-c938-4d1f-a551-b7a85737fca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:07.124872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 914/1820] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:07 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:07.128993 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1837c2-ff10-44b0-8e19-dee4a490a879 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:07.133818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82838b4a-c938-4d1f-a551-b7a85737fca3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:07.141183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a508107-39eb-4266-a404-81d398faf70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:07.142123 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a508107-39eb-4266-a404-81d398faf70d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:07.142455 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a508107-39eb-4266-a404-81d398faf70d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:07.142730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a508107-39eb-4266-a404-81d398faf70d admin admin] PATH_INFO: `/v3/users/696b0428efa84361be36a6c1efce69d8` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:07.143870 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1837c2-ff10-44b0-8e19-dee4a490a879 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:07.143999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a508107-39eb-4266-a404-81d398faf70d admin admin] RBAC: Authorizing `identity:delete_user(user_id=696b0428efa84361be36a6c1efce69d8)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:07.145807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a508107-39eb-4266-a404-81d398faf70d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:07.150367 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92c125-d77f-4309-8e16-5399837952dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:07.151427 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92c125-d77f-4309-8e16-5399837952dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:07.152645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92c125-d77f-4309-8e16-5399837952dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:07.153535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92c125-d77f-4309-8e16-5399837952dd admin admin] PATH_INFO: `/v3/projects/379c68903ed44ed5aa003f6e2616934e/users/c67d2a012577468a875730c74b174da0/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:07.155906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92c125-d77f-4309-8e16-5399837952dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=379c68903ed44ed5aa003f6e2616934e, user_id=c67d2a012577468a875730c74b174da0, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:07.159532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92c125-d77f-4309-8e16-5399837952dd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:07.176866 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 915/1821] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:07 2022] PUT /identity/v3/projects/379c68903ed44ed5aa003f6e2616934e/users/c67d2a012577468a875730c74b174da0/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:07.188165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca8f5d-141d-4158-90a0-b1b100f2ce7a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:07.188451 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca8f5d-141d-4158-90a0-b1b100f2ce7a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:07.188720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca8f5d-141d-4158-90a0-b1b100f2ce7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:07.206933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4a508107-39eb-4266-a404-81d398faf70d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '696b0428efa84361be36a6c1efce69d8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:07.215213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4a508107-39eb-4266-a404-81d398faf70d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '696b0428efa84361be36a6c1efce69d8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:07.221114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cdca8f5d-141d-4158-90a0-b1b100f2ce7a None None] MFA Rules not processed for user `c67d2a012577468a875730c74b174da0`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:07.222352 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cdca8f5d-141d-4158-90a0-b1b100f2ce7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:07.231671 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4a508107-39eb-4266-a404-81d398faf70d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '696b0428efa84361be36a6c1efce69d8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:07.232400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4a508107-39eb-4266-a404-81d398faf70d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '696b0428efa84361be36a6c1efce69d8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:07.241633 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-4a508107-39eb-4266-a404-81d398faf70d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '696b0428efa84361be36a6c1efce69d8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:07.252749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 907/1822] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:07 2022] DELETE /identity/v3/users/696b0428efa84361be36a6c1efce69d8 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:07.257610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f6534-0794-491b-ba70-26414f995b3d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:07.257976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f6534-0794-491b-ba70-26414f995b3d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:07.258276 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1f6534-0794-491b-ba70-26414f995b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:07.274009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:07.284182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3a1f6534-0794-491b-ba70-26414f995b3d None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:07.287057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3a1f6534-0794-491b-ba70-26414f995b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:07.317966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 916/1823] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:07.332483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca8f5d-141d-4158-90a0-b1b100f2ce7a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:07.334099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cdca8f5d-141d-4158-90a0-b1b100f2ce7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:07.367755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 908/1824] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:07.379037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f6534-0794-491b-ba70-26414f995b3d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:07.382133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3a1f6534-0794-491b-ba70-26414f995b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:07.393319 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:07.406085 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca8f5d-141d-4158-90a0-b1b100f2ce7a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:07.461586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1f6534-0794-491b-ba70-26414f995b3d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:07.471262 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2184cf7-0103-40c6-9204-65d6cc3eeeb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:07.472058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2184cf7-0103-40c6-9204-65d6cc3eeeb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:07.472492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2184cf7-0103-40c6-9204-65d6cc3eeeb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:07.472932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2184cf7-0103-40c6-9204-65d6cc3eeeb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:07.486106 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2184cf7-0103-40c6-9204-65d6cc3eeeb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:07.487608 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2184cf7-0103-40c6-9204-65d6cc3eeeb5 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:07.503118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7fb32-aa58-4684-9681-df87e843b6d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:07.503118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7fb32-aa58-4684-9681-df87e843b6d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:07.503118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7fb32-aa58-4684-9681-df87e843b6d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:07.503118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad7fb32-aa58-4684-9681-df87e843b6d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:07.503561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 909/1825] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:07.503660 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7790e6a-c201-470c-8a36-2d89156381bf None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:07.503890 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7790e6a-c201-470c-8a36-2d89156381bf None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:07.504293 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7790e6a-c201-470c-8a36-2d89156381bf None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:07.507809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 910/1826] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 15:27:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:27:07.526272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bc2c7-f035-4de4-b8aa-66b284b28a37 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:07.532466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bc2c7-f035-4de4-b8aa-66b284b28a37 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:07.532466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8bc2c7-f035-4de4-b8aa-66b284b28a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:07.559649 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cd8bc2c7-f035-4de4-b8aa-66b284b28a37 None None] MFA Rules not processed for user `fb745b98e3ae4723bad44428646f83e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:07.560941 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cd8bc2c7-f035-4de4-b8aa-66b284b28a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:07.585010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7fb32-aa58-4684-9681-df87e843b6d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:07.586864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad7fb32-aa58-4684-9681-df87e843b6d0 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:07.616394 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 917/1827] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:07.655650 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 911/1828] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 15:27:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:07.669864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7fb32-aa58-4684-9681-df87e843b6d0 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:07.672579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9ad7fb32-aa58-4684-9681-df87e843b6d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:07.691413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad7fb32-aa58-4684-9681-df87e843b6d0 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:07.698854 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5eb48-aac3-4cea-8aca-6ecf0b6dce70 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:07.699911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5eb48-aac3-4cea-8aca-6ecf0b6dce70 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:07.700463 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5eb48-aac3-4cea-8aca-6ecf0b6dce70 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:07.700665 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b5eb48-aac3-4cea-8aca-6ecf0b6dce70 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:07.708271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bc2c7-f035-4de4-b8aa-66b284b28a37 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:07.718029 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8bc2c7-f035-4de4-b8aa-66b284b28a37 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:07.718889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5eb48-aac3-4cea-8aca-6ecf0b6dce70 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:07.720851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b5eb48-aac3-4cea-8aca-6ecf0b6dce70 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:07.721736 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01866a-07d1-4947-89d0-cda0c8b68711 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:07.722379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01866a-07d1-4947-89d0-cda0c8b68711 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:07.722709 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01866a-07d1-4947-89d0-cda0c8b68711 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:07.722981 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d01866a-07d1-4947-89d0-cda0c8b68711 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:07.724969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0d01866a-07d1-4947-89d0-cda0c8b68711 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:07.736197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 918/1829] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:07.776729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:07.794629 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d01866a-07d1-4947-89d0-cda0c8b68711 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:07.796753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d01866a-07d1-4947-89d0-cda0c8b68711 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:07.815185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 912/1830] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:07.959632 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5eb48-aac3-4cea-8aca-6ecf0b6dce70 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:07.962437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-56b5eb48-aac3-4cea-8aca-6ecf0b6dce70 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:07.994614 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b5eb48-aac3-4cea-8aca-6ecf0b6dce70 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:08.005970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e6142-71b7-4f1a-8c22-96bc50b94189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:08.006712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e6142-71b7-4f1a-8c22-96bc50b94189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:08.006906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e6142-71b7-4f1a-8c22-96bc50b94189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:08.007175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53e6142-71b7-4f1a-8c22-96bc50b94189 admin admin] PATH_INFO: `/v3/projects/cb1ef28d7cff479196c6da7f907f0c2d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:08.013212 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e6142-71b7-4f1a-8c22-96bc50b94189 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb1ef28d7cff479196c6da7f907f0c2d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:08.015348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53e6142-71b7-4f1a-8c22-96bc50b94189 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:08.086452 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a53e6142-71b7-4f1a-8c22-96bc50b94189 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb1ef28d7cff479196c6da7f907f0c2d', 'request_id': 'req-a53e6142-71b7-4f1a-8c22-96bc50b94189'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:08.093365 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a53e6142-71b7-4f1a-8c22-96bc50b94189 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb1ef28d7cff479196c6da7f907f0c2d', 'request_id': 'req-a53e6142-71b7-4f1a-8c22-96bc50b94189'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:08.102423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a53e6142-71b7-4f1a-8c22-96bc50b94189 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb1ef28d7cff479196c6da7f907f0c2d', 'request_id': 'req-a53e6142-71b7-4f1a-8c22-96bc50b94189'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:08.115795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a53e6142-71b7-4f1a-8c22-96bc50b94189 admin admin] The token cache is being invalidate because project cb1ef28d7cff479196c6da7f907f0c2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:08.116085 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a53e6142-71b7-4f1a-8c22-96bc50b94189 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:08.116847 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a53e6142-71b7-4f1a-8c22-96bc50b94189 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:08.118605 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 919/1831] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:07 2022] DELETE /identity/v3/projects/cb1ef28d7cff479196c6da7f907f0c2d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:08.122531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01866a-07d1-4947-89d0-cda0c8b68711 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:08.134015 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d01866a-07d1-4947-89d0-cda0c8b68711 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:08.198880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd6db42-5889-4e84-a860-3d8f078731c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:08.199515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6db42-5889-4e84-a860-3d8f078731c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:08.199791 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6db42-5889-4e84-a860-3d8f078731c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:08.200065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd6db42-5889-4e84-a860-3d8f078731c6 admin admin] PATH_INFO: `/v3/users/0596914323ee4d7a8d3d67c40c0af2e3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:08.201204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6db42-5889-4e84-a860-3d8f078731c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0596914323ee4d7a8d3d67c40c0af2e3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:08.203796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd6db42-5889-4e84-a860-3d8f078731c6 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:08.264778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6dd6db42-5889-4e84-a860-3d8f078731c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0596914323ee4d7a8d3d67c40c0af2e3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:08.273009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6dd6db42-5889-4e84-a860-3d8f078731c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0596914323ee4d7a8d3d67c40c0af2e3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:08.286358 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6dd6db42-5889-4e84-a860-3d8f078731c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0596914323ee4d7a8d3d67c40c0af2e3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:08.287051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6dd6db42-5889-4e84-a860-3d8f078731c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0596914323ee4d7a8d3d67c40c0af2e3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:08.294045 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6dd6db42-5889-4e84-a860-3d8f078731c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0596914323ee4d7a8d3d67c40c0af2e3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:08.303005 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 913/1832] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:08 2022] DELETE /identity/v3/users/0596914323ee4d7a8d3d67c40c0af2e3 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:08.699522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e6142-71b7-4f1a-8c22-96bc50b94189 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:08.701066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a53e6142-71b7-4f1a-8c22-96bc50b94189 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:08.718484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53e6142-71b7-4f1a-8c22-96bc50b94189 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:08.745536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:08.771153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6644ee-c13e-4c35-8511-dbca7d22cae9 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:08.771774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6644ee-c13e-4c35-8511-dbca7d22cae9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:08.772041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6644ee-c13e-4c35-8511-dbca7d22cae9 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:08.773610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 914/1833] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 15:27:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 15:27:08.782695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34a87e-c78e-4017-a691-c716a677a663 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:08.783335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34a87e-c78e-4017-a691-c716a677a663 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:08.783335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f34a87e-c78e-4017-a691-c716a677a663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:08.811903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5f34a87e-c78e-4017-a691-c716a677a663 None None] MFA Rules not processed for user `fb745b98e3ae4723bad44428646f83e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:08.813633 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5f34a87e-c78e-4017-a691-c716a677a663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:08.817145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:08.817805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:08.818094 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:08.821843 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02 admin admin] PATH_INFO: `/v3/projects/fd026be13d63446ebefbdb226e76e022` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:08.823124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd026be13d63446ebefbdb226e76e022)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:08.825047 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:08.903864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd026be13d63446ebefbdb226e76e022', 'request_id': 'req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:08.907669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:08.911065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd026be13d63446ebefbdb226e76e022', 'request_id': 'req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:08.919283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 915/1834] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 15:27:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:08.920153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34a87e-c78e-4017-a691-c716a677a663 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:08.921329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd026be13d63446ebefbdb226e76e022', 'request_id': 'req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:08.923209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5f34a87e-c78e-4017-a691-c716a677a663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:08.935821 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02 admin admin] The token cache is being invalidate because project fd026be13d63446ebefbdb226e76e022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:08.936644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:08.937274 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:08.937844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f34a87e-c78e-4017-a691-c716a677a663 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:08.938798 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 920/1835] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:08 2022] DELETE /identity/v3/projects/fd026be13d63446ebefbdb226e76e022 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:08.939559 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:08.941918 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:09.000257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95199789-25e3-4e76-8559-d268a52b83e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:09.000831 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95199789-25e3-4e76-8559-d268a52b83e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:09.001136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95199789-25e3-4e76-8559-d268a52b83e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:09.001402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95199789-25e3-4e76-8559-d268a52b83e4 admin admin] PATH_INFO: `/v3/users/d848a95ccb434723b29fd47b57519b10` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:09.002575 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95199789-25e3-4e76-8559-d268a52b83e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d848a95ccb434723b29fd47b57519b10)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:09.004516 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95199789-25e3-4e76-8559-d268a52b83e4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:09.014430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81a43f-b596-4092-9c5d-e7ae6ca75b02 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:09.019479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f1f6a-22c8-4787-9ac4-609e1f1164d3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:09.020085 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f1f6a-22c8-4787-9ac4-609e1f1164d3 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:09.020400 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f1f6a-22c8-4787-9ac4-609e1f1164d3 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:09.020670 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f1f6a-22c8-4787-9ac4-609e1f1164d3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:09.022430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a81f1f6a-22c8-4787-9ac4-609e1f1164d3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:09.036698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f1f6a-22c8-4787-9ac4-609e1f1164d3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:09.038774 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f1f6a-22c8-4787-9ac4-609e1f1164d3 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:09.049951 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 921/1836] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:09.051248 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1166b1e-3454-4cb9-b153-cd8a17632cbd None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:09.051532 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1166b1e-3454-4cb9-b153-cd8a17632cbd None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:09.051805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1166b1e-3454-4cb9-b153-cd8a17632cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:09.069533 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-95199789-25e3-4e76-8559-d268a52b83e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd848a95ccb434723b29fd47b57519b10'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:09.078318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-95199789-25e3-4e76-8559-d268a52b83e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd848a95ccb434723b29fd47b57519b10'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:09.078486 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b1166b1e-3454-4cb9-b153-cd8a17632cbd None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:09.079645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b1166b1e-3454-4cb9-b153-cd8a17632cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:09.089725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 922/1837] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:09.092816 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-95199789-25e3-4e76-8559-d268a52b83e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd848a95ccb434723b29fd47b57519b10'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:09.093512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-95199789-25e3-4e76-8559-d268a52b83e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd848a95ccb434723b29fd47b57519b10'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:09.097472 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a35eb-eb36-4777-bb4a-1487f8db1de7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:09.097763 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a35eb-eb36-4777-bb4a-1487f8db1de7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:09.098037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317a35eb-eb36-4777-bb4a-1487f8db1de7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:09.101120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-95199789-25e3-4e76-8559-d268a52b83e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd848a95ccb434723b29fd47b57519b10'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:09.107047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 916/1838] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:08 2022] DELETE /identity/v3/users/d848a95ccb434723b29fd47b57519b10 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:09.117819 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-317a35eb-eb36-4777-bb4a-1487f8db1de7 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:09.119058 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-317a35eb-eb36-4777-bb4a-1487f8db1de7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:09.148733 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:09.184261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 923/1839] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:09.188583 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95199789-25e3-4e76-8559-d268a52b83e4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:09.199406 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95199789-25e3-4e76-8559-d268a52b83e4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:09.204661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcf313-21de-4135-b22f-3750bef8a431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:09.205265 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcf313-21de-4135-b22f-3750bef8a431 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:09.205540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcf313-21de-4135-b22f-3750bef8a431 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:09.205806 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dcf313-21de-4135-b22f-3750bef8a431 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:09.206596 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcf313-21de-4135-b22f-3750bef8a431 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:09.206911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcf313-21de-4135-b22f-3750bef8a431 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:09.208548 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dcf313-21de-4135-b22f-3750bef8a431 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:09.220966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 917/1840] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:09.225043 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a35eb-eb36-4777-bb4a-1487f8db1de7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:09.234685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317a35eb-eb36-4777-bb4a-1487f8db1de7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:09.239340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4cf86a-7399-4e55-a844-f8c3b941d919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:09.240057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cf86a-7399-4e55-a844-f8c3b941d919 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:09.240341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cf86a-7399-4e55-a844-f8c3b941d919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:09.240614 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4cf86a-7399-4e55-a844-f8c3b941d919 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:09.241274 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4cf86a-7399-4e55-a844-f8c3b941d919 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:09.243197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4cf86a-7399-4e55-a844-f8c3b941d919 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:09.274370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 924/1841] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:09 2022] POST /identity/v3/projects => generated 359 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:09.278318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcf313-21de-4135-b22f-3750bef8a431 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:09.289911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dcf313-21de-4135-b22f-3750bef8a431 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:09.290861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4cf86a-7399-4e55-a844-f8c3b941d919 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:09.292520 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7e4cf86a-7399-4e55-a844-f8c3b941d919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:09.361072 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4cf86a-7399-4e55-a844-f8c3b941d919 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:09.365894 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:09.366560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:09.366842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:09.367113 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab admin admin] PATH_INFO: `/v3/projects/4c97cc48060d4a4c99d8cd566250f972` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:09.368233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c97cc48060d4a4c99d8cd566250f972)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:09.369467 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a48e9-3ce5-41c2-b272-74a5445d23a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:09.370050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a48e9-3ce5-41c2-b272-74a5445d23a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:09.370149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:09.370374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a48e9-3ce5-41c2-b272-74a5445d23a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:09.370644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352a48e9-3ce5-41c2-b272-74a5445d23a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:09.371271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a48e9-3ce5-41c2-b272-74a5445d23a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:09.373183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352a48e9-3ce5-41c2-b272-74a5445d23a8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:09.384519 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:09.401462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 918/1842] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:09 2022] POST /identity/v3/users => generated 280 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:09.402787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a48e9-3ce5-41c2-b272-74a5445d23a8 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:09.415411 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352a48e9-3ce5-41c2-b272-74a5445d23a8 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:09.424298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c97cc48060d4a4c99d8cd566250f972', 'request_id': 'req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:09.429700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c97cc48060d4a4c99d8cd566250f972', 'request_id': 'req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:09.439250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c97cc48060d4a4c99d8cd566250f972', 'request_id': 'req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:09.450995 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab admin admin] The token cache is being invalidate because project 4c97cc48060d4a4c99d8cd566250f972 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:09.451280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:09.451982 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:09.453470 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 925/1843] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:09 2022] DELETE /identity/v3/projects/4c97cc48060d4a4c99d8cd566250f972 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:09.457435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:09.458834 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:09.476124 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6286c4d-3042-4c24-aa9b-633175fd86d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:09.476732 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6286c4d-3042-4c24-aa9b-633175fd86d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:09.477011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6286c4d-3042-4c24-aa9b-633175fd86d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:09.477515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6286c4d-3042-4c24-aa9b-633175fd86d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:09.478231 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6286c4d-3042-4c24-aa9b-633175fd86d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:09.479870 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6286c4d-3042-4c24-aa9b-633175fd86d3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:09.489963 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 919/1844] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:09.493936 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6286c4d-3042-4c24-aa9b-633175fd86d3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:09.495678 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b6286c4d-3042-4c24-aa9b-633175fd86d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:09.508631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:09.508798 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6286c4d-3042-4c24-aa9b-633175fd86d3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:09.514152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862f7a2-b703-48b3-9db4-1363db11cbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:09.514921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862f7a2-b703-48b3-9db4-1363db11cbd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:09.515233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862f7a2-b703-48b3-9db4-1363db11cbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:09.515530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0862f7a2-b703-48b3-9db4-1363db11cbd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:09.516168 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862f7a2-b703-48b3-9db4-1363db11cbd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:09.518001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0862f7a2-b703-48b3-9db4-1363db11cbd2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:09.529110 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 920/1845] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:09.533598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862f7a2-b703-48b3-9db4-1363db11cbd2 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:09.537248 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ff251a-16f8-410e-9fe7-8aa92ebc77ab admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:09.542401 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e095a-fc36-4951-96b2-0f82320c1aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:09.543046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e095a-fc36-4951-96b2-0f82320c1aa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:09.543326 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e095a-fc36-4951-96b2-0f82320c1aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:09.543594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e095a-fc36-4951-96b2-0f82320c1aa1 admin admin] PATH_INFO: `/v3/users/5b1b7484ec094b6e8422d04fce4d878e` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:09.544343 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0862f7a2-b703-48b3-9db4-1363db11cbd2 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:09.544829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e095a-fc36-4951-96b2-0f82320c1aa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b1b7484ec094b6e8422d04fce4d878e)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:09.547498 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e095a-fc36-4951-96b2-0f82320c1aa1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:09.549956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca3f7cf-889f-4914-bbcf-79e67f073905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:09.550696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3f7cf-889f-4914-bbcf-79e67f073905 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:09.551045 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3f7cf-889f-4914-bbcf-79e67f073905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:09.551385 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3f7cf-889f-4914-bbcf-79e67f073905 admin admin] PATH_INFO: `/v3/projects/846b8609db7142bda78ac744f0863472/users/11770265aad344488f5f2a3f877cb420/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:09.568948 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca3f7cf-889f-4914-bbcf-79e67f073905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=846b8609db7142bda78ac744f0863472, user_id=11770265aad344488f5f2a3f877cb420, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:09.569497 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca3f7cf-889f-4914-bbcf-79e67f073905 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:09.583279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 921/1846] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:09 2022] PUT /identity/v3/projects/846b8609db7142bda78ac744f0863472/users/11770265aad344488f5f2a3f877cb420/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:09.594234 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce10f1-537e-433c-830f-bd556c84cbba None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:09.594549 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce10f1-537e-433c-830f-bd556c84cbba None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:09.594813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce10f1-537e-433c-830f-bd556c84cbba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:09.602404 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0d5e095a-fc36-4951-96b2-0f82320c1aa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b1b7484ec094b6e8422d04fce4d878e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:09.603275 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0d5e095a-fc36-4951-96b2-0f82320c1aa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b1b7484ec094b6e8422d04fce4d878e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:09.616055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0d5e095a-fc36-4951-96b2-0f82320c1aa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b1b7484ec094b6e8422d04fce4d878e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:09.622538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f5ce10f1-537e-433c-830f-bd556c84cbba None None] MFA Rules not processed for user `11770265aad344488f5f2a3f877cb420`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:09.624451 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0d5e095a-fc36-4951-96b2-0f82320c1aa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b1b7484ec094b6e8422d04fce4d878e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:09.624566 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f5ce10f1-537e-433c-830f-bd556c84cbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:09.625330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0d5e095a-fc36-4951-96b2-0f82320c1aa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b1b7484ec094b6e8422d04fce4d878e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:09.637931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 926/1847] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:09 2022] DELETE /identity/v3/users/5b1b7484ec094b6e8422d04fce4d878e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:09.691610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 922/1848] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:09 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:09.734586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e095a-fc36-4951-96b2-0f82320c1aa1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:09.736077 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0d5e095a-fc36-4951-96b2-0f82320c1aa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:09.796364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e095a-fc36-4951-96b2-0f82320c1aa1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:09.800185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c28f2-ba0e-42b5-a343-75f39c414031 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:09.800824 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c28f2-ba0e-42b5-a343-75f39c414031 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:09.801102 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c28f2-ba0e-42b5-a343-75f39c414031 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:09.801373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1c28f2-ba0e-42b5-a343-75f39c414031 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:09.832193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce10f1-537e-433c-830f-bd556c84cbba None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:09.834182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f5ce10f1-537e-433c-830f-bd556c84cbba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:09.866064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c28f2-ba0e-42b5-a343-75f39c414031 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:09.867550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1c28f2-ba0e-42b5-a343-75f39c414031 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:09.876899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 927/1849] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:09 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:09.892890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c28f2-ba0e-42b5-a343-75f39c414031 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:09.894351 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4e1c28f2-ba0e-42b5-a343-75f39c414031 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:09.920871 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:09.925683 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:09.957013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce10f1-537e-433c-830f-bd556c84cbba None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:10.002564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1c28f2-ba0e-42b5-a343-75f39c414031 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:10.009675 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87149e64-3213-442c-abdb-f08402af4a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:10.010519 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87149e64-3213-442c-abdb-f08402af4a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.010888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87149e64-3213-442c-abdb-f08402af4a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.011286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87149e64-3213-442c-abdb-f08402af4a9d admin admin] PATH_INFO: `/v3/users/11770265aad344488f5f2a3f877cb420` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.013434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87149e64-3213-442c-abdb-f08402af4a9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=11770265aad344488f5f2a3f877cb420)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:10.016035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87149e64-3213-442c-abdb-f08402af4a9d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:10.026807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52b503e-6c24-430f-863a-cb1fddb805e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:10.027456 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b503e-6c24-430f-863a-cb1fddb805e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.027737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b503e-6c24-430f-863a-cb1fddb805e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.028013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52b503e-6c24-430f-863a-cb1fddb805e6 admin admin] PATH_INFO: `/v3/projects/f2667661fd1f4e0bb91b453d32236915` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.029181 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b503e-6c24-430f-863a-cb1fddb805e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2667661fd1f4e0bb91b453d32236915)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:10.031264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52b503e-6c24-430f-863a-cb1fddb805e6 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:10.092018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e52b503e-6c24-430f-863a-cb1fddb805e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2667661fd1f4e0bb91b453d32236915', 'request_id': 'req-e52b503e-6c24-430f-863a-cb1fddb805e6'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:10.106316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-87149e64-3213-442c-abdb-f08402af4a9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11770265aad344488f5f2a3f877cb420'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:10.106678 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e52b503e-6c24-430f-863a-cb1fddb805e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2667661fd1f4e0bb91b453d32236915', 'request_id': 'req-e52b503e-6c24-430f-863a-cb1fddb805e6'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:10.106769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-87149e64-3213-442c-abdb-f08402af4a9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11770265aad344488f5f2a3f877cb420'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:10.113645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e52b503e-6c24-430f-863a-cb1fddb805e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2667661fd1f4e0bb91b453d32236915', 'request_id': 'req-e52b503e-6c24-430f-863a-cb1fddb805e6'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:10.121015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-87149e64-3213-442c-abdb-f08402af4a9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11770265aad344488f5f2a3f877cb420'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:10.127176 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e52b503e-6c24-430f-863a-cb1fddb805e6 admin admin] The token cache is being invalidate because project f2667661fd1f4e0bb91b453d32236915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:10.127516 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e52b503e-6c24-430f-863a-cb1fddb805e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:10.128241 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e52b503e-6c24-430f-863a-cb1fddb805e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:10.129413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-87149e64-3213-442c-abdb-f08402af4a9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11770265aad344488f5f2a3f877cb420'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:10.130044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 923/1850] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:09 2022] DELETE /identity/v3/projects/f2667661fd1f4e0bb91b453d32236915 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:10.130213 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-87149e64-3213-442c-abdb-f08402af4a9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11770265aad344488f5f2a3f877cb420'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:10.134734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015b2b5-5777-4a07-8356-e9166fb9b7e1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.135014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015b2b5-5777-4a07-8356-e9166fb9b7e1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.135282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015b2b5-5777-4a07-8356-e9166fb9b7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.143245 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 928/1851] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:09 2022] DELETE /identity/v3/users/11770265aad344488f5f2a3f877cb420 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:10.147578 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88ad74-32bd-470e-9333-333ec9448031 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.147859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88ad74-32bd-470e-9333-333ec9448031 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.148127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a88ad74-32bd-470e-9333-333ec9448031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.156686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6015b2b5-5777-4a07-8356-e9166fb9b7e1 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:10.158197 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6015b2b5-5777-4a07-8356-e9166fb9b7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:10.167034 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3a88ad74-32bd-470e-9333-333ec9448031 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:10.168190 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3a88ad74-32bd-470e-9333-333ec9448031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:10.221508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 924/1852] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:10.229131 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc369c-cd7a-4247-833f-75ff9de7c8c1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.229431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc369c-cd7a-4247-833f-75ff9de7c8c1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.229703 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc369c-cd7a-4247-833f-75ff9de7c8c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.234350 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 929/1853] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:10.243845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88ad74-32bd-470e-9333-333ec9448031 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:10.245183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3a88ad74-32bd-470e-9333-333ec9448031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:10.249591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-26fc369c-cd7a-4247-833f-75ff9de7c8c1 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:10.250811 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-26fc369c-cd7a-4247-833f-75ff9de7c8c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:10.259143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 925/1854] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:10.263305 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc369c-cd7a-4247-833f-75ff9de7c8c1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:10.274747 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc369c-cd7a-4247-833f-75ff9de7c8c1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:10.280114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1322f5-582f-43e8-bc59-cc39a7b7bebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:10.280809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1322f5-582f-43e8-bc59-cc39a7b7bebb admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.281094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1322f5-582f-43e8-bc59-cc39a7b7bebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.281433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1322f5-582f-43e8-bc59-cc39a7b7bebb admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.282317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1322f5-582f-43e8-bc59-cc39a7b7bebb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:10.282626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1322f5-582f-43e8-bc59-cc39a7b7bebb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:10.284400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1322f5-582f-43e8-bc59-cc39a7b7bebb admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:10.289907 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:10.296694 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 926/1855] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:27:10.301177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1322f5-582f-43e8-bc59-cc39a7b7bebb admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:10.307336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:10.310754 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a88ad74-32bd-470e-9333-333ec9448031 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:10.312073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1322f5-582f-43e8-bc59-cc39a7b7bebb admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:10.315235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66ab76-95c8-4eb7-9d0d-db83df650e3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:10.317090 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66ab76-95c8-4eb7-9d0d-db83df650e3d service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.317090 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66ab76-95c8-4eb7-9d0d-db83df650e3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.317090 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f66ab76-95c8-4eb7-9d0d-db83df650e3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.319932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d546f09-51ed-4388-a74a-33e6efccb8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:10.320550 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d546f09-51ed-4388-a74a-33e6efccb8f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.320876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d546f09-51ed-4388-a74a-33e6efccb8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.321135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d546f09-51ed-4388-a74a-33e6efccb8f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.321789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d546f09-51ed-4388-a74a-33e6efccb8f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:10.323788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d546f09-51ed-4388-a74a-33e6efccb8f8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:10.328126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f66ab76-95c8-4eb7-9d0d-db83df650e3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:10.329551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f66ab76-95c8-4eb7-9d0d-db83df650e3d service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:10.339649 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 930/1856] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:10.352310 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 927/1857] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:10 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:10.356427 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66ab76-95c8-4eb7-9d0d-db83df650e3d service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:10.366126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f66ab76-95c8-4eb7-9d0d-db83df650e3d service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:10.430071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ed137-cd34-4e23-aec7-f8485d12f087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:10.430071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ed137-cd34-4e23-aec7-f8485d12f087 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.430071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ed137-cd34-4e23-aec7-f8485d12f087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.430071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3ed137-cd34-4e23-aec7-f8485d12f087 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.430772 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ed137-cd34-4e23-aec7-f8485d12f087 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:10.432975 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3ed137-cd34-4e23-aec7-f8485d12f087 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:10.455666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 931/1858] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:10 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:10.459839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d546f09-51ed-4388-a74a-33e6efccb8f8 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:10.471293 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d546f09-51ed-4388-a74a-33e6efccb8f8 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:10.476397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5adf5-b74b-42a1-89fb-a974137b7074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:10.477078 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5adf5-b74b-42a1-89fb-a974137b7074 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.477431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5adf5-b74b-42a1-89fb-a974137b7074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.477818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b5adf5-b74b-42a1-89fb-a974137b7074 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.478661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5adf5-b74b-42a1-89fb-a974137b7074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:10.480300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b5adf5-b74b-42a1-89fb-a974137b7074 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:10.491170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 928/1859] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:10.498330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ed137-cd34-4e23-aec7-f8485d12f087 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:10.509190 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3ed137-cd34-4e23-aec7-f8485d12f087 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:10.513973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964a2683-1107-462b-833a-151e6d6e4766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:10.515172 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a2683-1107-462b-833a-151e6d6e4766 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.515467 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a2683-1107-462b-833a-151e6d6e4766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.515737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964a2683-1107-462b-833a-151e6d6e4766 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.516394 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a2683-1107-462b-833a-151e6d6e4766 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:10.517991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964a2683-1107-462b-833a-151e6d6e4766 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:10.528517 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 932/1860] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:10.533192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5adf5-b74b-42a1-89fb-a974137b7074 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:10.546106 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b5adf5-b74b-42a1-89fb-a974137b7074 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:10.550518 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857cafa-3879-4a8d-8447-b8521ef87595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:10.551181 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857cafa-3879-4a8d-8447-b8521ef87595 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.551764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857cafa-3879-4a8d-8447-b8521ef87595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.551764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857cafa-3879-4a8d-8447-b8521ef87595 admin admin] PATH_INFO: `/v3/projects/d12d49c49967427e971bf9d64f93de6a/users/1250a8d8b1494e07b48e0515f70e865c/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.567634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857cafa-3879-4a8d-8447-b8521ef87595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d12d49c49967427e971bf9d64f93de6a, user_id=1250a8d8b1494e07b48e0515f70e865c, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:10.570579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857cafa-3879-4a8d-8447-b8521ef87595 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:10.590327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 929/1861] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:10 2022] PUT /identity/v3/projects/d12d49c49967427e971bf9d64f93de6a/users/1250a8d8b1494e07b48e0515f70e865c/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:10.601974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4ed39-58cd-48cb-97aa-b41078ee2db4 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.602421 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4ed39-58cd-48cb-97aa-b41078ee2db4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.606375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c4ed39-58cd-48cb-97aa-b41078ee2db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.633302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-46c4ed39-58cd-48cb-97aa-b41078ee2db4 None None] MFA Rules not processed for user `1250a8d8b1494e07b48e0515f70e865c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:10.633799 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857cafa-3879-4a8d-8447-b8521ef87595 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:10.634593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-46c4ed39-58cd-48cb-97aa-b41078ee2db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:10.635813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4857cafa-3879-4a8d-8447-b8521ef87595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:10.709451 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857cafa-3879-4a8d-8447-b8521ef87595 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:10.710766 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 933/1862] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:10.714149 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6afd2-7d71-4404-8502-e0cc87ff9952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:10.714824 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6afd2-7d71-4404-8502-e0cc87ff9952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.715105 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6afd2-7d71-4404-8502-e0cc87ff9952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.715197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4ed39-58cd-48cb-97aa-b41078ee2db4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:10.715381 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6afd2-7d71-4404-8502-e0cc87ff9952 admin admin] PATH_INFO: `/v3/projects/846b8609db7142bda78ac744f0863472` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.716658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6afd2-7d71-4404-8502-e0cc87ff9952 admin admin] RBAC: Authorizing `identity:delete_project(project_id=846b8609db7142bda78ac744f0863472)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:10.719212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6afd2-7d71-4404-8502-e0cc87ff9952 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:10.725973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c4ed39-58cd-48cb-97aa-b41078ee2db4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:10.730525 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd8c60-216f-4194-9e38-87f0e731c410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:10.731111 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd8c60-216f-4194-9e38-87f0e731c410 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.731382 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd8c60-216f-4194-9e38-87f0e731c410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.731654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdd8c60-216f-4194-9e38-87f0e731c410 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.732262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd8c60-216f-4194-9e38-87f0e731c410 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:10.734153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdd8c60-216f-4194-9e38-87f0e731c410 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:10.762273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 934/1863] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:10 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:10.766037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd8c60-216f-4194-9e38-87f0e731c410 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:10.769577 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fcd6afd2-7d71-4404-8502-e0cc87ff9952 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '846b8609db7142bda78ac744f0863472', 'request_id': 'req-fcd6afd2-7d71-4404-8502-e0cc87ff9952'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:10.776062 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fcd6afd2-7d71-4404-8502-e0cc87ff9952 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '846b8609db7142bda78ac744f0863472', 'request_id': 'req-fcd6afd2-7d71-4404-8502-e0cc87ff9952'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:10.786293 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fcd6afd2-7d71-4404-8502-e0cc87ff9952 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '846b8609db7142bda78ac744f0863472', 'request_id': 'req-fcd6afd2-7d71-4404-8502-e0cc87ff9952'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:10.797937 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:10.800243 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fcd6afd2-7d71-4404-8502-e0cc87ff9952 admin admin] The token cache is being invalidate because project 846b8609db7142bda78ac744f0863472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:10.800641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fcd6afd2-7d71-4404-8502-e0cc87ff9952 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:10.801507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fcd6afd2-7d71-4404-8502-e0cc87ff9952 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:10.803241 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 930/1864] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:10 2022] DELETE /identity/v3/projects/846b8609db7142bda78ac744f0863472 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:10.808685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8641c79-3d95-4943-91b2-16e6efd579ce None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.808979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8641c79-3d95-4943-91b2-16e6efd579ce None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.809249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8641c79-3d95-4943-91b2-16e6efd579ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.833215 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b8641c79-3d95-4943-91b2-16e6efd579ce None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:10.834364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b8641c79-3d95-4943-91b2-16e6efd579ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:10.850253 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdd8c60-216f-4194-9e38-87f0e731c410 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:10.855145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1821fb0c-212c-43f3-af48-2a9d98104bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:10.855759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1821fb0c-212c-43f3-af48-2a9d98104bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.856043 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1821fb0c-212c-43f3-af48-2a9d98104bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.856323 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1821fb0c-212c-43f3-af48-2a9d98104bd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.856992 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1821fb0c-212c-43f3-af48-2a9d98104bd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:10.859476 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1821fb0c-212c-43f3-af48-2a9d98104bd4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:10.870210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:10.886290 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 935/1865] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:10 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:10.887797 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1821fb0c-212c-43f3-af48-2a9d98104bd4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:10.890091 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1821fb0c-212c-43f3-af48-2a9d98104bd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:10.904255 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1821fb0c-212c-43f3-af48-2a9d98104bd4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:10.905943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 931/1866] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:10.910731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a827f-062b-469c-95be-866b8a41a308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:10.911421 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a827f-062b-469c-95be-866b8a41a308 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.911735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a827f-062b-469c-95be-866b8a41a308 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.912045 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9a827f-062b-469c-95be-866b8a41a308 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.912717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a827f-062b-469c-95be-866b8a41a308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:10.915191 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9a827f-062b-469c-95be-866b8a41a308 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:10.915789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9696f09-081a-4679-880d-c7bef16c1d32 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.916280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9696f09-081a-4679-880d-c7bef16c1d32 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.916720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9696f09-081a-4679-880d-c7bef16c1d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.934582 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 936/1867] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:10.935701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a827f-062b-469c-95be-866b8a41a308 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:10.941583 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c9696f09-081a-4679-880d-c7bef16c1d32 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:10.942975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c9696f09-081a-4679-880d-c7bef16c1d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:10.947630 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9a827f-062b-469c-95be-866b8a41a308 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:10.953494 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ef5ba-00ab-49bb-8cdc-0accdf7ffabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:10.954069 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 932/1868] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:10.954256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ef5ba-00ab-49bb-8cdc-0accdf7ffabd admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.954481 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ef5ba-00ab-49bb-8cdc-0accdf7ffabd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.954759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859ef5ba-00ab-49bb-8cdc-0accdf7ffabd admin admin] PATH_INFO: `/v3/projects/4e522b2f25554e7b8971641de39d9091/users/cd4c15ea1f784edaa0bcf2a5b1133e43/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.959317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9696f09-081a-4679-880d-c7bef16c1d32 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:10.969271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9696f09-081a-4679-880d-c7bef16c1d32 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:10.971630 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ef5ba-00ab-49bb-8cdc-0accdf7ffabd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e522b2f25554e7b8971641de39d9091, user_id=cd4c15ea1f784edaa0bcf2a5b1133e43, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:10.973823 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eefa60-eda3-4305-afb7-0f5765d32ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:10.974425 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859ef5ba-00ab-49bb-8cdc-0accdf7ffabd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:10.974555 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eefa60-eda3-4305-afb7-0f5765d32ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:10.974707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eefa60-eda3-4305-afb7-0f5765d32ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:10.974973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7eefa60-eda3-4305-afb7-0f5765d32ed0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:10.975638 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eefa60-eda3-4305-afb7-0f5765d32ed0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:10.975938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eefa60-eda3-4305-afb7-0f5765d32ed0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:10.977508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7eefa60-eda3-4305-afb7-0f5765d32ed0 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:10.989903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 937/1869] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:10 2022] PUT /identity/v3/projects/4e522b2f25554e7b8971641de39d9091/users/cd4c15ea1f784edaa0bcf2a5b1133e43/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:10.993025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 933/1870] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:10.993768 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ef5ba-00ab-49bb-8cdc-0accdf7ffabd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:10.997099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eefa60-eda3-4305-afb7-0f5765d32ed0 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:11.008176 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7eefa60-eda3-4305-afb7-0f5765d32ed0 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:11.062242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859ef5ba-00ab-49bb-8cdc-0accdf7ffabd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:11.067124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ccacd-9c5c-40fc-9e2d-6d32e6b1d82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:11.067846 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ccacd-9c5c-40fc-9e2d-6d32e6b1d82c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:11.068248 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ccacd-9c5c-40fc-9e2d-6d32e6b1d82c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:11.068583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669ccacd-9c5c-40fc-9e2d-6d32e6b1d82c admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:11.069352 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ccacd-9c5c-40fc-9e2d-6d32e6b1d82c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:11.071109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669ccacd-9c5c-40fc-9e2d-6d32e6b1d82c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:11.074879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f85311-ba21-4977-896e-b15d8716f24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:11.075734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f85311-ba21-4977-896e-b15d8716f24f admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:11.076263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f85311-ba21-4977-896e-b15d8716f24f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:11.076870 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f85311-ba21-4977-896e-b15d8716f24f admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:11.078818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f85311-ba21-4977-896e-b15d8716f24f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:11.081272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f85311-ba21-4977-896e-b15d8716f24f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:11.083689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 938/1871] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:11.089000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ccacd-9c5c-40fc-9e2d-6d32e6b1d82c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:11.100499 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669ccacd-9c5c-40fc-9e2d-6d32e6b1d82c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:11.106999 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a686a1f0-abc2-4a18-b0fa-481f5f5ec906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:11.107741 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686a1f0-abc2-4a18-b0fa-481f5f5ec906 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:11.108106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686a1f0-abc2-4a18-b0fa-481f5f5ec906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:11.108445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a686a1f0-abc2-4a18-b0fa-481f5f5ec906 admin admin] PATH_INFO: `/v3/projects/4e522b2f25554e7b8971641de39d9091/users/cd4c15ea1f784edaa0bcf2a5b1133e43/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:11.111053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686a1f0-abc2-4a18-b0fa-481f5f5ec906 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e522b2f25554e7b8971641de39d9091, user_id=cd4c15ea1f784edaa0bcf2a5b1133e43, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:11.113992 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a686a1f0-abc2-4a18-b0fa-481f5f5ec906 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:11.136048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 934/1872] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:10 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:11.140669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f85311-ba21-4977-896e-b15d8716f24f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:11.146932 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 939/1873] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:11 2022] PUT /identity/v3/projects/4e522b2f25554e7b8971641de39d9091/users/cd4c15ea1f784edaa0bcf2a5b1133e43/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:11.158727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe2d15-8c5b-494a-b56c-0721bcaa210c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:11.159028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe2d15-8c5b-494a-b56c-0721bcaa210c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:11.159289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe2d15-8c5b-494a-b56c-0721bcaa210c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:11.210995 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-edfe2d15-8c5b-494a-b56c-0721bcaa210c None None] MFA Rules not processed for user `cd4c15ea1f784edaa0bcf2a5b1133e43`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:11.212327 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-edfe2d15-8c5b-494a-b56c-0721bcaa210c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:11.253457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f85311-ba21-4977-896e-b15d8716f24f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:11.260677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e06d71-7b22-451c-ab04-5ee200ac92f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:11.261377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e06d71-7b22-451c-ab04-5ee200ac92f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:11.261731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e06d71-7b22-451c-ab04-5ee200ac92f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:11.262073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e06d71-7b22-451c-ab04-5ee200ac92f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:11.262805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e06d71-7b22-451c-ab04-5ee200ac92f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:11.266069 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e06d71-7b22-451c-ab04-5ee200ac92f1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:11.301980 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 935/1874] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:11 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:11.303126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e06d71-7b22-451c-ab04-5ee200ac92f1 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:11.329348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:11.330099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e06d71-7b22-451c-ab04-5ee200ac92f1 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:11.337883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d349c7-7eb4-420d-8787-2b02dedf5bf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:11.338561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d349c7-7eb4-420d-8787-2b02dedf5bf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:11.338841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d349c7-7eb4-420d-8787-2b02dedf5bf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:11.339117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d349c7-7eb4-420d-8787-2b02dedf5bf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:11.339828 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d349c7-7eb4-420d-8787-2b02dedf5bf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:11.341785 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d349c7-7eb4-420d-8787-2b02dedf5bf0 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:11.362528 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 936/1875] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:11.366264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 940/1876] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:11.369790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d349c7-7eb4-420d-8787-2b02dedf5bf0 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:11.377238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe2d15-8c5b-494a-b56c-0721bcaa210c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:11.378674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-edfe2d15-8c5b-494a-b56c-0721bcaa210c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:11.380953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d349c7-7eb4-420d-8787-2b02dedf5bf0 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:11.386471 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c6fd2-02ba-4e5f-8cd4-0c153f840ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:11.387078 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c6fd2-02ba-4e5f-8cd4-0c153f840ac7 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:11.387359 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c6fd2-02ba-4e5f-8cd4-0c153f840ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:11.387631 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c6fd2-02ba-4e5f-8cd4-0c153f840ac7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:11.388272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c6fd2-02ba-4e5f-8cd4-0c153f840ac7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:11.390449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468c6fd2-02ba-4e5f-8cd4-0c153f840ac7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:11.404619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 937/1877] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:11.409271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c6fd2-02ba-4e5f-8cd4-0c153f840ac7 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:11.420197 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c6fd2-02ba-4e5f-8cd4-0c153f840ac7 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:11.425730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9f0085-ad5c-4cfd-8d77-877236da51b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:11.426455 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f0085-ad5c-4cfd-8d77-877236da51b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:11.426733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f0085-ad5c-4cfd-8d77-877236da51b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:11.427003 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f0085-ad5c-4cfd-8d77-877236da51b9 admin admin] PATH_INFO: `/v3/projects/ee689c95ffec4cdabd0c5fd83c82a753/users/170dd6f119354f3dab51eb962bb032d7/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:11.435949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:11.445398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe2d15-8c5b-494a-b56c-0721bcaa210c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:11.446165 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f0085-ad5c-4cfd-8d77-877236da51b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee689c95ffec4cdabd0c5fd83c82a753, user_id=170dd6f119354f3dab51eb962bb032d7, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:11.449422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9f0085-ad5c-4cfd-8d77-877236da51b9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:11.449539 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e00730-cfaa-438e-a64e-a57b868b1bf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:11.450136 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e00730-cfaa-438e-a64e-a57b868b1bf4 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:11.450446 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e00730-cfaa-438e-a64e-a57b868b1bf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:11.450772 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e00730-cfaa-438e-a64e-a57b868b1bf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:11.463826 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e00730-cfaa-438e-a64e-a57b868b1bf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:11.465205 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e00730-cfaa-438e-a64e-a57b868b1bf4 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:11.466636 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 938/1878] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:11 2022] PUT /identity/v3/projects/ee689c95ffec4cdabd0c5fd83c82a753/users/170dd6f119354f3dab51eb962bb032d7/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:11.479773 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 941/1879] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:11.479871 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7f1df-eb02-45ef-835c-ad6914711269 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:11.479871 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7f1df-eb02-45ef-835c-ad6914711269 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:11.479871 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b7f1df-eb02-45ef-835c-ad6914711269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:11.515385 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f6b7f1df-eb02-45ef-835c-ad6914711269 None None] MFA Rules not processed for user `170dd6f119354f3dab51eb962bb032d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:11.516667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f6b7f1df-eb02-45ef-835c-ad6914711269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:11.568465 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e00730-cfaa-438e-a64e-a57b868b1bf4 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:11.576918 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e00730-cfaa-438e-a64e-a57b868b1bf4 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:11.603342 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 939/1880] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:11.607689 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7f1df-eb02-45ef-835c-ad6914711269 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:11.640480 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc81c24-ac55-4753-8e73-287f21415df4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:11.641292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc81c24-ac55-4753-8e73-287f21415df4 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:11.641750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc81c24-ac55-4753-8e73-287f21415df4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:11.642181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc81c24-ac55-4753-8e73-287f21415df4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:11.644108 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fbc81c24-ac55-4753-8e73-287f21415df4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:11.699361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b7f1df-eb02-45ef-835c-ad6914711269 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:11.704709 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ea78e-b420-4211-a840-83ef86f8b4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:11.705354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ea78e-b420-4211-a840-83ef86f8b4b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:11.705748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ea78e-b420-4211-a840-83ef86f8b4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:11.706047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ea78e-b420-4211-a840-83ef86f8b4b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:11.706711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ea78e-b420-4211-a840-83ef86f8b4b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:11.708667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ea78e-b420-4211-a840-83ef86f8b4b5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:11.721267 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc81c24-ac55-4753-8e73-287f21415df4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:11.723032 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc81c24-ac55-4753-8e73-287f21415df4 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:11.732663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 942/1881] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:11.745993 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 940/1882] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:11 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:11.750589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc81c24-ac55-4753-8e73-287f21415df4 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:11.762680 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc81c24-ac55-4753-8e73-287f21415df4 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:11.820217 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:11.846677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e62a0-c112-43c7-9c8c-d7a896cb5d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:11.847501 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e62a0-c112-43c7-9c8c-d7a896cb5d8c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:11.847710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e62a0-c112-43c7-9c8c-d7a896cb5d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:11.847987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14e62a0-c112-43c7-9c8c-d7a896cb5d8c admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:11.848703 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e62a0-c112-43c7-9c8c-d7a896cb5d8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:11.850730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14e62a0-c112-43c7-9c8c-d7a896cb5d8c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:11.877413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 943/1883] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:11 2022] POST /identity/v3/users => generated 289 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:11.882757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ea78e-b420-4211-a840-83ef86f8b4b5 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:11.896958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ea78e-b420-4211-a840-83ef86f8b4b5 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:11.903806 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c941e1e-40d3-4fca-b6ab-ddf90a78ccae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:11.904420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c941e1e-40d3-4fca-b6ab-ddf90a78ccae admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:11.905417 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c941e1e-40d3-4fca-b6ab-ddf90a78ccae admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:11.905704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c941e1e-40d3-4fca-b6ab-ddf90a78ccae admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:11.906540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c941e1e-40d3-4fca-b6ab-ddf90a78ccae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:11.908665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c941e1e-40d3-4fca-b6ab-ddf90a78ccae admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:11.926259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 941/1884] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:11.928901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e62a0-c112-43c7-9c8c-d7a896cb5d8c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:11.939892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14e62a0-c112-43c7-9c8c-d7a896cb5d8c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:11.947579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fb55d4-3672-460a-a18a-0fd96f85dce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:11.948437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb55d4-3672-460a-a18a-0fd96f85dce0 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:11.948752 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb55d4-3672-460a-a18a-0fd96f85dce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:11.948946 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fb55d4-3672-460a-a18a-0fd96f85dce0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:11.949617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb55d4-3672-460a-a18a-0fd96f85dce0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:11.951322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fb55d4-3672-460a-a18a-0fd96f85dce0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:11.965038 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 944/1885] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:11.971652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c941e1e-40d3-4fca-b6ab-ddf90a78ccae admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:11.981728 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c941e1e-40d3-4fca-b6ab-ddf90a78ccae admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:11.989335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859bf58-b1f9-442a-b6c4-0302484289d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:11.990015 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859bf58-b1f9-442a-b6c4-0302484289d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:11.990327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859bf58-b1f9-442a-b6c4-0302484289d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:11.990610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859bf58-b1f9-442a-b6c4-0302484289d9 admin admin] PATH_INFO: `/v3/projects/8e51610cbd714848a333d8f1b1ef73cf/users/3c165cf8aba64721a9bcd47a80809d63/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:12.007694 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859bf58-b1f9-442a-b6c4-0302484289d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e51610cbd714848a333d8f1b1ef73cf, user_id=3c165cf8aba64721a9bcd47a80809d63, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:12.011553 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859bf58-b1f9-442a-b6c4-0302484289d9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:12.028919 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 942/1886] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:11 2022] PUT /identity/v3/projects/8e51610cbd714848a333d8f1b1ef73cf/users/3c165cf8aba64721a9bcd47a80809d63/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:12.041421 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945e307-0269-4e54-b08a-253ef5afc178 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:12.041758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945e307-0269-4e54-b08a-253ef5afc178 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:12.042005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945e307-0269-4e54-b08a-253ef5afc178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:12.075646 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4945e307-0269-4e54-b08a-253ef5afc178 None None] MFA Rules not processed for user `3c165cf8aba64721a9bcd47a80809d63`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:12.077502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4945e307-0269-4e54-b08a-253ef5afc178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:12.154285 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 945/1887] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:12.159393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859bf58-b1f9-442a-b6c4-0302484289d9 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:12.181056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945e307-0269-4e54-b08a-253ef5afc178 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:12.182710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4945e307-0269-4e54-b08a-253ef5afc178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:12.219022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:12.265757 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945e307-0269-4e54-b08a-253ef5afc178 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:12.265960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859bf58-b1f9-442a-b6c4-0302484289d9 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:12.270716 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebe069-111c-4b8b-9084-0ffb33859f22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:12.271407 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebe069-111c-4b8b-9084-0ffb33859f22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:12.271516 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5446a-11c0-4f45-a5d4-9eff71ed9d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:12.271688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebe069-111c-4b8b-9084-0ffb33859f22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:12.271955 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebe069-111c-4b8b-9084-0ffb33859f22 admin admin] PATH_INFO: `/v3/users/b40f119e1cc44bffb9e3554ea473a417` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:12.272122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5446a-11c0-4f45-a5d4-9eff71ed9d83 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:12.272404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5446a-11c0-4f45-a5d4-9eff71ed9d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:12.272676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e5446a-11c0-4f45-a5d4-9eff71ed9d83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:12.273254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebe069-111c-4b8b-9084-0ffb33859f22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b40f119e1cc44bffb9e3554ea473a417)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:12.273347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e5446a-11c0-4f45-a5d4-9eff71ed9d83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:12.275274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e5446a-11c0-4f45-a5d4-9eff71ed9d83 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:12.275802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebe069-111c-4b8b-9084-0ffb33859f22 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:12.303677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 943/1888] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:12 2022] POST /identity/v3/projects => generated 377 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:12.304515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5446a-11c0-4f45-a5d4-9eff71ed9d83 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:12.306063 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-56e5446a-11c0-4f45-a5d4-9eff71ed9d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:12.331317 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e1ebe069-111c-4b8b-9084-0ffb33859f22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b40f119e1cc44bffb9e3554ea473a417'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:12.332082 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e1ebe069-111c-4b8b-9084-0ffb33859f22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b40f119e1cc44bffb9e3554ea473a417'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:12.341132 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:12.355525 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e1ebe069-111c-4b8b-9084-0ffb33859f22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b40f119e1cc44bffb9e3554ea473a417'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:12.366471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e1ebe069-111c-4b8b-9084-0ffb33859f22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b40f119e1cc44bffb9e3554ea473a417'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:12.367161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e1ebe069-111c-4b8b-9084-0ffb33859f22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b40f119e1cc44bffb9e3554ea473a417'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:12.380496 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e5446a-11c0-4f45-a5d4-9eff71ed9d83 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:12.384578 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 946/1889] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:12 2022] DELETE /identity/v3/users/b40f119e1cc44bffb9e3554ea473a417 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:12.385416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f5600-6c13-4686-90eb-8d87742eef04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:12.385565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebe069-111c-4b8b-9084-0ffb33859f22 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:12.386057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f5600-6c13-4686-90eb-8d87742eef04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:12.386361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f5600-6c13-4686-90eb-8d87742eef04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:12.386633 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f5600-6c13-4686-90eb-8d87742eef04 admin admin] PATH_INFO: `/v3/users/6047d21a2f624d3b9c9b1d3852a39848` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:12.387876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f5600-6c13-4686-90eb-8d87742eef04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6047d21a2f624d3b9c9b1d3852a39848)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:12.388520 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e1ebe069-111c-4b8b-9084-0ffb33859f22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:12.389859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363f5600-6c13-4686-90eb-8d87742eef04 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:12.477547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-363f5600-6c13-4686-90eb-8d87742eef04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6047d21a2f624d3b9c9b1d3852a39848'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:12.494595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-363f5600-6c13-4686-90eb-8d87742eef04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6047d21a2f624d3b9c9b1d3852a39848'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:12.494802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebe069-111c-4b8b-9084-0ffb33859f22 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:12.506614 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1c42a-79ae-4b2e-a179-7f9ea670425b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:12.507043 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-363f5600-6c13-4686-90eb-8d87742eef04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6047d21a2f624d3b9c9b1d3852a39848'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:12.507900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-363f5600-6c13-4686-90eb-8d87742eef04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6047d21a2f624d3b9c9b1d3852a39848'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:12.508256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1c42a-79ae-4b2e-a179-7f9ea670425b admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:12.508710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1c42a-79ae-4b2e-a179-7f9ea670425b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:12.509126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1c42a-79ae-4b2e-a179-7f9ea670425b admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:12.510647 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1c42a-79ae-4b2e-a179-7f9ea670425b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:12.516336 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef1c42a-79ae-4b2e-a179-7f9ea670425b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:12.524447 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-363f5600-6c13-4686-90eb-8d87742eef04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6047d21a2f624d3b9c9b1d3852a39848'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:12.538325 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 944/1890] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:12 2022] DELETE /identity/v3/users/6047d21a2f624d3b9c9b1d3852a39848 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:12.539702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b23a3d-106a-4eaf-8364-806f2fbaecee None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:12.539999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b23a3d-106a-4eaf-8364-806f2fbaecee None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:12.540295 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b23a3d-106a-4eaf-8364-806f2fbaecee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:12.568120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-56b23a3d-106a-4eaf-8364-806f2fbaecee None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:12.571541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-56b23a3d-106a-4eaf-8364-806f2fbaecee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:12.585570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 947/1891] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:12 2022] POST /identity/v3/users => generated 295 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:12.587341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b917491-98c3-4820-a3b9-c3a24e5f07a4 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:12.587341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b917491-98c3-4820-a3b9-c3a24e5f07a4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:12.587566 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b917491-98c3-4820-a3b9-c3a24e5f07a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:12.610087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:12.626301 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5b917491-98c3-4820-a3b9-c3a24e5f07a4 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:12.627947 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5b917491-98c3-4820-a3b9-c3a24e5f07a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:12.658017 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 945/1892] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:12.659022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b23a3d-106a-4eaf-8364-806f2fbaecee None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:12.660454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-56b23a3d-106a-4eaf-8364-806f2fbaecee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:12.674927 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b23a3d-106a-4eaf-8364-806f2fbaecee None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:12.679802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df5d6c-0bcd-4206-ad71-90edc6b0653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:12.680498 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df5d6c-0bcd-4206-ad71-90edc6b0653a admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:12.681294 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df5d6c-0bcd-4206-ad71-90edc6b0653a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:12.681294 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df5d6c-0bcd-4206-ad71-90edc6b0653a admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:12.681879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df5d6c-0bcd-4206-ad71-90edc6b0653a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:12.683631 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df5d6c-0bcd-4206-ad71-90edc6b0653a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:12.699116 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 946/1893] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:12.700183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df5d6c-0bcd-4206-ad71-90edc6b0653a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:12.703816 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-39df5d6c-0bcd-4206-ad71-90edc6b0653a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:12.721082 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 948/1894] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:12.721808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b917491-98c3-4820-a3b9-c3a24e5f07a4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:12.734035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b917491-98c3-4820-a3b9-c3a24e5f07a4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:12.741838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d4edc-3706-449b-9f48-1e6eaad5b4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:12.742701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d4edc-3706-449b-9f48-1e6eaad5b4b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:12.742980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d4edc-3706-449b-9f48-1e6eaad5b4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:12.743260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d4edc-3706-449b-9f48-1e6eaad5b4b2 admin admin] PATH_INFO: `/v3/projects/e18f121e4ac74e3a916aa09b3174e53c/users/81bebea949bf4df3acf117abd9e45349/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:12.759655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d4edc-3706-449b-9f48-1e6eaad5b4b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18f121e4ac74e3a916aa09b3174e53c, user_id=81bebea949bf4df3acf117abd9e45349, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:12.763571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d4edc-3706-449b-9f48-1e6eaad5b4b2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:12.771897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df5d6c-0bcd-4206-ad71-90edc6b0653a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:12.781330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 949/1895] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:12 2022] PUT /identity/v3/projects/e18f121e4ac74e3a916aa09b3174e53c/users/81bebea949bf4df3acf117abd9e45349/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:12.782555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d4edc-3706-449b-9f48-1e6eaad5b4b2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:12.785839 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ac8d4edc-3706-449b-9f48-1e6eaad5b4b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:12.831434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:12.839282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c1124-a863-49b4-a977-83a2bc00b74f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:12.839951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c1124-a863-49b4-a977-83a2bc00b74f service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:12.840268 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c1124-a863-49b4-a977-83a2bc00b74f service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:12.840701 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c1124-a863-49b4-a977-83a2bc00b74f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:12.853783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d4edc-3706-449b-9f48-1e6eaad5b4b2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:12.857595 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fed78c-5664-4acb-84df-9c080750269d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:12.858788 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fed78c-5664-4acb-84df-9c080750269d service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:12.859059 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fed78c-5664-4acb-84df-9c080750269d service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:12.859322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fed78c-5664-4acb-84df-9c080750269d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:12.906672 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c1124-a863-49b4-a977-83a2bc00b74f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:12.908382 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c1124-a863-49b4-a977-83a2bc00b74f service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:12.918947 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 947/1896] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:12.919577 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c1124-a863-49b4-a977-83a2bc00b74f service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:12.922752 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fed78c-5664-4acb-84df-9c080750269d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:12.924349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fed78c-5664-4acb-84df-9c080750269d service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:12.931143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c1124-a863-49b4-a977-83a2bc00b74f service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:12.939922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 950/1897] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:12.940260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1084fbc1-f9a3-43a3-adff-0f6bcc902818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:12.941496 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084fbc1-f9a3-43a3-adff-0f6bcc902818 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:12.941802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084fbc1-f9a3-43a3-adff-0f6bcc902818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:12.943152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1084fbc1-f9a3-43a3-adff-0f6bcc902818 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:12.944140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084fbc1-f9a3-43a3-adff-0f6bcc902818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:12.946707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1084fbc1-f9a3-43a3-adff-0f6bcc902818 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:12.964960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 948/1898] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:12.966148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fed78c-5664-4acb-84df-9c080750269d service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:12.978867 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fed78c-5664-4acb-84df-9c080750269d service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:12.987695 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d76184-23f9-46e9-bc82-e4047641ed9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:12.988595 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d76184-23f9-46e9-bc82-e4047641ed9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:12.989002 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d76184-23f9-46e9-bc82-e4047641ed9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:12.989235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d76184-23f9-46e9-bc82-e4047641ed9f admin admin] PATH_INFO: `/v3/projects/e18f121e4ac74e3a916aa09b3174e53c/users/81bebea949bf4df3acf117abd9e45349/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:12.992150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d76184-23f9-46e9-bc82-e4047641ed9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e18f121e4ac74e3a916aa09b3174e53c, user_id=81bebea949bf4df3acf117abd9e45349, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:12.995436 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d76184-23f9-46e9-bc82-e4047641ed9f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:13.012487 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 951/1899] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:12 2022] PUT /identity/v3/projects/e18f121e4ac74e3a916aa09b3174e53c/users/81bebea949bf4df3acf117abd9e45349/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:13.024310 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1790e1c-e9b0-4bd1-95b9-6cd0c88b0ad5 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:13.024725 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1790e1c-e9b0-4bd1-95b9-6cd0c88b0ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:13.024830 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1790e1c-e9b0-4bd1-95b9-6cd0c88b0ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:13.064313 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a1790e1c-e9b0-4bd1-95b9-6cd0c88b0ad5 None None] MFA Rules not processed for user `81bebea949bf4df3acf117abd9e45349`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:13.065965 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a1790e1c-e9b0-4bd1-95b9-6cd0c88b0ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:13.071579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:13.165869 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 949/1900] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:13 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:13.182182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d76184-23f9-46e9-bc82-e4047641ed9f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:13.183996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-74d76184-23f9-46e9-bc82-e4047641ed9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:13.260693 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d76184-23f9-46e9-bc82-e4047641ed9f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:13.265916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665e5e8-5a35-4499-9432-4b9339f13506 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:13.266990 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665e5e8-5a35-4499-9432-4b9339f13506 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:13.267512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665e5e8-5a35-4499-9432-4b9339f13506 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:13.267926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665e5e8-5a35-4499-9432-4b9339f13506 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:13.271039 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9665e5e8-5a35-4499-9432-4b9339f13506 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:13.313478 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1790e1c-e9b0-4bd1-95b9-6cd0c88b0ad5 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:13.315154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a1790e1c-e9b0-4bd1-95b9-6cd0c88b0ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:13.369837 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665e5e8-5a35-4499-9432-4b9339f13506 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:13.371666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665e5e8-5a35-4499-9432-4b9339f13506 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:13.384879 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 952/1901] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:13 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:13.386068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665e5e8-5a35-4499-9432-4b9339f13506 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:13.387716 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9665e5e8-5a35-4499-9432-4b9339f13506 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:13.438819 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:13.440056 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1790e1c-e9b0-4bd1-95b9-6cd0c88b0ad5 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:13.456032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d2c26-f2f7-4c26-b37f-83709356869d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:13.456032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2c26-f2f7-4c26-b37f-83709356869d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:13.456032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2c26-f2f7-4c26-b37f-83709356869d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:13.456032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d2c26-f2f7-4c26-b37f-83709356869d admin admin] PATH_INFO: `/v3/projects/be998fc12a3141c58c68248858161ea7` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:13.459640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d2c26-f2f7-4c26-b37f-83709356869d admin admin] RBAC: Authorizing `identity:delete_project(project_id=be998fc12a3141c58c68248858161ea7)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:13.459640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d2c26-f2f7-4c26-b37f-83709356869d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:13.517478 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665e5e8-5a35-4499-9432-4b9339f13506 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:13.535286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cf1d2c26-f2f7-4c26-b37f-83709356869d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be998fc12a3141c58c68248858161ea7', 'request_id': 'req-cf1d2c26-f2f7-4c26-b37f-83709356869d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:13.551208 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cf1d2c26-f2f7-4c26-b37f-83709356869d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be998fc12a3141c58c68248858161ea7', 'request_id': 'req-cf1d2c26-f2f7-4c26-b37f-83709356869d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:13.565983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cf1d2c26-f2f7-4c26-b37f-83709356869d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be998fc12a3141c58c68248858161ea7', 'request_id': 'req-cf1d2c26-f2f7-4c26-b37f-83709356869d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:13.601494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cf1d2c26-f2f7-4c26-b37f-83709356869d admin admin] The token cache is being invalidate because project be998fc12a3141c58c68248858161ea7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:13.601900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cf1d2c26-f2f7-4c26-b37f-83709356869d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:13.604045 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cf1d2c26-f2f7-4c26-b37f-83709356869d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:13.606865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 950/1902] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:13 2022] DELETE /identity/v3/projects/be998fc12a3141c58c68248858161ea7 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:13.612200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d2c26-f2f7-4c26-b37f-83709356869d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:13.616953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cf1d2c26-f2f7-4c26-b37f-83709356869d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:13.664672 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:13.687072 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:13.687778 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:13.688060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:13.689178 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776 admin admin] PATH_INFO: `/v3/projects/8f3d9ae76f4b408da4ab498d3405f36b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:13.690343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f3d9ae76f4b408da4ab498d3405f36b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:13.692760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:13.741984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d2c26-f2f7-4c26-b37f-83709356869d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:13.809188 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f3d9ae76f4b408da4ab498d3405f36b', 'request_id': 'req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:13.827963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f3d9ae76f4b408da4ab498d3405f36b', 'request_id': 'req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:13.838566 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f3d9ae76f4b408da4ab498d3405f36b', 'request_id': 'req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:13.872937 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776 admin admin] The token cache is being invalidate because project 8f3d9ae76f4b408da4ab498d3405f36b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:13.873876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:13.884366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:13.886353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 953/1903] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:13 2022] DELETE /identity/v3/projects/8f3d9ae76f4b408da4ab498d3405f36b => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:13.892156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:13.894553 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:13.930437 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e3caf-28e4-4fdb-b44d-94593072cafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:13.932000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e3caf-28e4-4fdb-b44d-94593072cafa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:13.932000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e3caf-28e4-4fdb-b44d-94593072cafa admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:13.932000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344e3caf-28e4-4fdb-b44d-94593072cafa admin admin] PATH_INFO: `/v3/users/ef34cfe262b44f1a8b307e06d33db2e1` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:13.934392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e3caf-28e4-4fdb-b44d-94593072cafa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef34cfe262b44f1a8b307e06d33db2e1)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:13.937502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344e3caf-28e4-4fdb-b44d-94593072cafa admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:14.015621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734d1bcc-4b68-44fb-a0e3-2f6a2b548776 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:14.021821 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4da13-f990-4c45-8fda-d1737e00323e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:14.022456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4da13-f990-4c45-8fda-d1737e00323e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:14.023041 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4da13-f990-4c45-8fda-d1737e00323e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:14.023381 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec4da13-f990-4c45-8fda-d1737e00323e admin admin] PATH_INFO: `/v3/users/c67d2a012577468a875730c74b174da0` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:14.024618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4da13-f990-4c45-8fda-d1737e00323e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c67d2a012577468a875730c74b174da0)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:14.026993 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec4da13-f990-4c45-8fda-d1737e00323e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:14.047970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:14.064477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-344e3caf-28e4-4fdb-b44d-94593072cafa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef34cfe262b44f1a8b307e06d33db2e1'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.076670 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-344e3caf-28e4-4fdb-b44d-94593072cafa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef34cfe262b44f1a8b307e06d33db2e1'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.097370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-344e3caf-28e4-4fdb-b44d-94593072cafa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef34cfe262b44f1a8b307e06d33db2e1'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.098468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-344e3caf-28e4-4fdb-b44d-94593072cafa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef34cfe262b44f1a8b307e06d33db2e1'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.108118 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-344e3caf-28e4-4fdb-b44d-94593072cafa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef34cfe262b44f1a8b307e06d33db2e1'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.119642 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 951/1904] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:13 2022] DELETE /identity/v3/users/ef34cfe262b44f1a8b307e06d33db2e1 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:14.135333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3ec4da13-f990-4c45-8fda-d1737e00323e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c67d2a012577468a875730c74b174da0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.137023 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3ec4da13-f990-4c45-8fda-d1737e00323e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c67d2a012577468a875730c74b174da0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.174115 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3ec4da13-f990-4c45-8fda-d1737e00323e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c67d2a012577468a875730c74b174da0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.185727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3ec4da13-f990-4c45-8fda-d1737e00323e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c67d2a012577468a875730c74b174da0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.187139 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3ec4da13-f990-4c45-8fda-d1737e00323e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c67d2a012577468a875730c74b174da0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.213306 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 954/1905] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:13 2022] DELETE /identity/v3/users/c67d2a012577468a875730c74b174da0 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:14.483480 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e3caf-28e4-4fdb-b44d-94593072cafa admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:14.485176 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-344e3caf-28e4-4fdb-b44d-94593072cafa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:14.563078 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:14.567377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344e3caf-28e4-4fdb-b44d-94593072cafa admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:14.628736 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c833c8-dc55-4472-8af0-aa4c1c1a3aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:14.629260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c833c8-dc55-4472-8af0-aa4c1c1a3aac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:14.629535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c833c8-dc55-4472-8af0-aa4c1c1a3aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:14.629809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c833c8-dc55-4472-8af0-aa4c1c1a3aac admin admin] PATH_INFO: `/v3/projects/c22b4103e8b540e39ef54e606c18f6ab` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:14.630986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c833c8-dc55-4472-8af0-aa4c1c1a3aac admin admin] RBAC: Authorizing `identity:delete_project(project_id=c22b4103e8b540e39ef54e606c18f6ab)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:14.632909 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c833c8-dc55-4472-8af0-aa4c1c1a3aac admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:14.693117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a4c833c8-dc55-4472-8af0-aa4c1c1a3aac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c22b4103e8b540e39ef54e606c18f6ab', 'request_id': 'req-a4c833c8-dc55-4472-8af0-aa4c1c1a3aac'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.700187 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a4c833c8-dc55-4472-8af0-aa4c1c1a3aac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c22b4103e8b540e39ef54e606c18f6ab', 'request_id': 'req-a4c833c8-dc55-4472-8af0-aa4c1c1a3aac'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.714377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a4c833c8-dc55-4472-8af0-aa4c1c1a3aac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c22b4103e8b540e39ef54e606c18f6ab', 'request_id': 'req-a4c833c8-dc55-4472-8af0-aa4c1c1a3aac'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.735869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4da13-f990-4c45-8fda-d1737e00323e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:14.736038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a4c833c8-dc55-4472-8af0-aa4c1c1a3aac admin admin] The token cache is being invalidate because project c22b4103e8b540e39ef54e606c18f6ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:14.736290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a4c833c8-dc55-4472-8af0-aa4c1c1a3aac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.737063 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a4c833c8-dc55-4472-8af0-aa4c1c1a3aac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.737974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3ec4da13-f990-4c45-8fda-d1737e00323e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:14.739234 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 952/1906] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:14 2022] DELETE /identity/v3/projects/c22b4103e8b540e39ef54e606c18f6ab => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:14.744019 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83726d9a-e8a5-4630-9b85-69486f99b1c6 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:14.744362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83726d9a-e8a5-4630-9b85-69486f99b1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:14.744706 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83726d9a-e8a5-4630-9b85-69486f99b1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:14.764029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec4da13-f990-4c45-8fda-d1737e00323e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:14.768633 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-83726d9a-e8a5-4630-9b85-69486f99b1c6 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:14.769942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-83726d9a-e8a5-4630-9b85-69486f99b1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:14.861984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa9564c-c5ec-4548-8a5d-fb19d1aea91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:14.862817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9564c-c5ec-4548-8a5d-fb19d1aea91e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:14.863198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9564c-c5ec-4548-8a5d-fb19d1aea91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:14.863603 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa9564c-c5ec-4548-8a5d-fb19d1aea91e admin admin] PATH_INFO: `/v3/projects/379c68903ed44ed5aa003f6e2616934e` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:14.865110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9564c-c5ec-4548-8a5d-fb19d1aea91e admin admin] RBAC: Authorizing `identity:delete_project(project_id=379c68903ed44ed5aa003f6e2616934e)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:14.868373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa9564c-c5ec-4548-8a5d-fb19d1aea91e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:14.883785 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 953/1907] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:14.884891 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3c941-6bb7-45e8-b03c-cbef1a79d7db None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:14.885221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3c941-6bb7-45e8-b03c-cbef1a79d7db None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:14.885511 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3c941-6bb7-45e8-b03c-cbef1a79d7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:14.910373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-66d3c941-6bb7-45e8-b03c-cbef1a79d7db None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:14.912143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-66d3c941-6bb7-45e8-b03c-cbef1a79d7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:14.929215 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 954/1908] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:14.930354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3c941-6bb7-45e8-b03c-cbef1a79d7db None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:14.944809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3c941-6bb7-45e8-b03c-cbef1a79d7db None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:14.963413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-caa9564c-c5ec-4548-8a5d-fb19d1aea91e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '379c68903ed44ed5aa003f6e2616934e', 'request_id': 'req-caa9564c-c5ec-4548-8a5d-fb19d1aea91e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.972560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-caa9564c-c5ec-4548-8a5d-fb19d1aea91e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '379c68903ed44ed5aa003f6e2616934e', 'request_id': 'req-caa9564c-c5ec-4548-8a5d-fb19d1aea91e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:14.988206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-caa9564c-c5ec-4548-8a5d-fb19d1aea91e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '379c68903ed44ed5aa003f6e2616934e', 'request_id': 'req-caa9564c-c5ec-4548-8a5d-fb19d1aea91e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:15.004166 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-caa9564c-c5ec-4548-8a5d-fb19d1aea91e admin admin] The token cache is being invalidate because project 379c68903ed44ed5aa003f6e2616934e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:15.004549 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-caa9564c-c5ec-4548-8a5d-fb19d1aea91e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:15.005429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-caa9564c-c5ec-4548-8a5d-fb19d1aea91e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:15.007364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 955/1909] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:14 2022] DELETE /identity/v3/projects/379c68903ed44ed5aa003f6e2616934e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:15.014365 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74d240-48a9-4edc-8e68-ebbba3b2180b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.015246 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74d240-48a9-4edc-8e68-ebbba3b2180b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.017554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d74d240-48a9-4edc-8e68-ebbba3b2180b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.023224 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:15.045618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9d74d240-48a9-4edc-8e68-ebbba3b2180b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:15.047659 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9d74d240-48a9-4edc-8e68-ebbba3b2180b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:15.079341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:15.079840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217ec7b-07ea-498f-8d12-71a67f25cf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:15.080912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217ec7b-07ea-498f-8d12-71a67f25cf40 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.081707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217ec7b-07ea-498f-8d12-71a67f25cf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.082388 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b217ec7b-07ea-498f-8d12-71a67f25cf40 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.083389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217ec7b-07ea-498f-8d12-71a67f25cf40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:15.083774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217ec7b-07ea-498f-8d12-71a67f25cf40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:15.085549 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b217ec7b-07ea-498f-8d12-71a67f25cf40 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:15.113231 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 955/1910] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:15.114987 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217ec7b-07ea-498f-8d12-71a67f25cf40 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:15.122755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b217ec7b-07ea-498f-8d12-71a67f25cf40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:15.146529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b217ec7b-07ea-498f-8d12-71a67f25cf40 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:15.154515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff1dd7-7df6-48a5-8bda-a856a3641d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:15.156025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff1dd7-7df6-48a5-8bda-a856a3641d7f admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.156025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff1dd7-7df6-48a5-8bda-a856a3641d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.158019 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff1dd7-7df6-48a5-8bda-a856a3641d7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.158710 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff1dd7-7df6-48a5-8bda-a856a3641d7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:15.161792 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff1dd7-7df6-48a5-8bda-a856a3641d7f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:15.172782 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 956/1911] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:15.180363 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722a500-cef0-4005-abbe-b7d9470306e5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.181914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722a500-cef0-4005-abbe-b7d9470306e5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.182616 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5722a500-cef0-4005-abbe-b7d9470306e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.196079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 956/1912] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:15 2022] POST /identity/v3/projects => generated 377 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:15.207750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff1dd7-7df6-48a5-8bda-a856a3641d7f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:15.224906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5722a500-cef0-4005-abbe-b7d9470306e5 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:15.226301 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5722a500-cef0-4005-abbe-b7d9470306e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:15.306574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 957/1913] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:15.306574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722a500-cef0-4005-abbe-b7d9470306e5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:15.313833 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5722a500-cef0-4005-abbe-b7d9470306e5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:15.319272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfe575-24f3-4375-9b09-6d6891382e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:15.319890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfe575-24f3-4375-9b09-6d6891382e69 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.320168 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfe575-24f3-4375-9b09-6d6891382e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.320439 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdfe575-24f3-4375-9b09-6d6891382e69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.321090 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfe575-24f3-4375-9b09-6d6891382e69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:15.321390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfe575-24f3-4375-9b09-6d6891382e69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:15.323094 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdfe575-24f3-4375-9b09-6d6891382e69 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:15.324837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff1dd7-7df6-48a5-8bda-a856a3641d7f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:15.332660 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dffeea-0cf2-424a-8f61-263202dd56e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:15.333840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dffeea-0cf2-424a-8f61-263202dd56e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.334390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dffeea-0cf2-424a-8f61-263202dd56e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.335030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dffeea-0cf2-424a-8f61-263202dd56e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.336057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 958/1914] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:15.336433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dffeea-0cf2-424a-8f61-263202dd56e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:15.338611 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dffeea-0cf2-424a-8f61-263202dd56e6 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:15.340650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfe575-24f3-4375-9b09-6d6891382e69 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:15.352675 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdfe575-24f3-4375-9b09-6d6891382e69 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:15.358905 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787cdd6-9821-407e-b16e-f42fa6d41c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:15.359518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787cdd6-9821-407e-b16e-f42fa6d41c1c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.359851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787cdd6-9821-407e-b16e-f42fa6d41c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.360065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b787cdd6-9821-407e-b16e-f42fa6d41c1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.360977 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787cdd6-9821-407e-b16e-f42fa6d41c1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:15.362934 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b787cdd6-9821-407e-b16e-f42fa6d41c1c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:15.382500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 957/1915] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:15 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:15.384362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dffeea-0cf2-424a-8f61-263202dd56e6 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:15.395757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dffeea-0cf2-424a-8f61-263202dd56e6 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:15.396887 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 959/1916] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:15 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:15.397725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787cdd6-9821-407e-b16e-f42fa6d41c1c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:15.399459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b787cdd6-9821-407e-b16e-f42fa6d41c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:15.460866 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b787cdd6-9821-407e-b16e-f42fa6d41c1c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:15.464977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df824d7a-dd85-4583-8ad8-abfd77aa43ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:15.465535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df824d7a-dd85-4583-8ad8-abfd77aa43ab admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.465659 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183b0f1-f886-4d4d-a135-73022077ef86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:15.465822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df824d7a-dd85-4583-8ad8-abfd77aa43ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.466095 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df824d7a-dd85-4583-8ad8-abfd77aa43ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.466295 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183b0f1-f886-4d4d-a135-73022077ef86 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.466582 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183b0f1-f886-4d4d-a135-73022077ef86 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.466781 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df824d7a-dd85-4583-8ad8-abfd77aa43ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:15.466880 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5183b0f1-f886-4d4d-a135-73022077ef86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.468399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df824d7a-dd85-4583-8ad8-abfd77aa43ab admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:15.468583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5183b0f1-f886-4d4d-a135-73022077ef86 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:15.481978 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 958/1917] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:15.481978 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df824d7a-dd85-4583-8ad8-abfd77aa43ab admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:15.490313 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df824d7a-dd85-4583-8ad8-abfd77aa43ab admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:15.495344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826004a0-32a8-416f-bb39-5d8658d5b916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:15.495937 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826004a0-32a8-416f-bb39-5d8658d5b916 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.496218 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826004a0-32a8-416f-bb39-5d8658d5b916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.496488 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826004a0-32a8-416f-bb39-5d8658d5b916 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.497178 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826004a0-32a8-416f-bb39-5d8658d5b916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:15.499605 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826004a0-32a8-416f-bb39-5d8658d5b916 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:15.508133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:15.525779 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 959/1918] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:15 2022] POST /identity/v3/users => generated 294 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:15.527099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826004a0-32a8-416f-bb39-5d8658d5b916 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:15.536863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183b0f1-f886-4d4d-a135-73022077ef86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:15.538019 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826004a0-32a8-416f-bb39-5d8658d5b916 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:15.538331 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5183b0f1-f886-4d4d-a135-73022077ef86 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:15.543121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb90552-2a04-482b-a364-e94bcfcf52a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:15.543758 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb90552-2a04-482b-a364-e94bcfcf52a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.544041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb90552-2a04-482b-a364-e94bcfcf52a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.544322 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb90552-2a04-482b-a364-e94bcfcf52a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.545047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb90552-2a04-482b-a364-e94bcfcf52a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:15.546759 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb90552-2a04-482b-a364-e94bcfcf52a2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:15.547672 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 960/1919] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:15.548470 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183b0f1-f886-4d4d-a135-73022077ef86 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:15.549115 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:15.560164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 960/1920] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:15.560304 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5183b0f1-f886-4d4d-a135-73022077ef86 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:15.565144 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb90552-2a04-482b-a364-e94bcfcf52a2 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:15.566299 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc236fe-b939-40d5-9d9d-3f060e3b720b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:15.566708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc236fe-b939-40d5-9d9d-3f060e3b720b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.566988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc236fe-b939-40d5-9d9d-3f060e3b720b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.567259 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc236fe-b939-40d5-9d9d-3f060e3b720b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.567900 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc236fe-b939-40d5-9d9d-3f060e3b720b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:15.569474 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc236fe-b939-40d5-9d9d-3f060e3b720b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:15.578257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb90552-2a04-482b-a364-e94bcfcf52a2 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:15.586729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 961/1921] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:15.587706 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771285e-a173-4348-a511-50f0d6da5b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:15.588137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc236fe-b939-40d5-9d9d-3f060e3b720b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:15.588527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771285e-a173-4348-a511-50f0d6da5b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.589005 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771285e-a173-4348-a511-50f0d6da5b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.589340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771285e-a173-4348-a511-50f0d6da5b51 admin admin] PATH_INFO: `/v3/projects/1abd098239644177ac3fda4d204ea6e1/users/c28048fb141449c3ae13c07069660a39/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.601223 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc236fe-b939-40d5-9d9d-3f060e3b720b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:15.606453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771285e-a173-4348-a511-50f0d6da5b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1abd098239644177ac3fda4d204ea6e1, user_id=c28048fb141449c3ae13c07069660a39, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:15.606809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db5660-b2b6-470f-81c9-616493a12752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:15.608297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db5660-b2b6-470f-81c9-616493a12752 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.608580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db5660-b2b6-470f-81c9-616493a12752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.608856 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db5660-b2b6-470f-81c9-616493a12752 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.609502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db5660-b2b6-470f-81c9-616493a12752 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:15.609636 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771285e-a173-4348-a511-50f0d6da5b51 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:15.611421 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db5660-b2b6-470f-81c9-616493a12752 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:15.624790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 962/1922] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:15.629620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db5660-b2b6-470f-81c9-616493a12752 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:15.638789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 961/1923] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:15 2022] PUT /identity/v3/projects/1abd098239644177ac3fda4d204ea6e1/users/c28048fb141449c3ae13c07069660a39/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:15.641090 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb16e9-ea56-46a1-932e-76767fcb8ce0 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.641809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb16e9-ea56-46a1-932e-76767fcb8ce0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.641809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bb16e9-ea56-46a1-932e-76767fcb8ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.649699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db5660-b2b6-470f-81c9-616493a12752 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:15.676541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-11bb16e9-ea56-46a1-932e-76767fcb8ce0 None None] MFA Rules not processed for user `c28048fb141449c3ae13c07069660a39`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:15.678380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-11bb16e9-ea56-46a1-932e-76767fcb8ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:15.717909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c65e49-34d5-437b-aeaf-46d369e405a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:15.718754 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c65e49-34d5-437b-aeaf-46d369e405a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.719023 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c65e49-34d5-437b-aeaf-46d369e405a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.719294 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c65e49-34d5-437b-aeaf-46d369e405a9 admin admin] PATH_INFO: `/v3/projects/b2e760e99a364299b9b3af03d6a2addd/users/63558284bd8242a8b149b44ffb7d34fd/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.736369 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c65e49-34d5-437b-aeaf-46d369e405a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2e760e99a364299b9b3af03d6a2addd, user_id=63558284bd8242a8b149b44ffb7d34fd, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:15.739109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c65e49-34d5-437b-aeaf-46d369e405a9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:15.753950 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 963/1924] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:15 2022] PUT /identity/v3/projects/b2e760e99a364299b9b3af03d6a2addd/users/63558284bd8242a8b149b44ffb7d34fd/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:15.760521 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 962/1925] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:15.765510 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb16e9-ea56-46a1-932e-76767fcb8ce0 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:15.765998 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4afbe-6aa2-4ebd-8407-cd4df4b83ab7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.766273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4afbe-6aa2-4ebd-8407-cd4df4b83ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.766576 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc4afbe-6aa2-4ebd-8407-cd4df4b83ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.776810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bb16e9-ea56-46a1-932e-76767fcb8ce0 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:15.796810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3cc4afbe-6aa2-4ebd-8407-cd4df4b83ab7 None None] MFA Rules not processed for user `63558284bd8242a8b149b44ffb7d34fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:15.798020 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3cc4afbe-6aa2-4ebd-8407-cd4df4b83ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:15.849305 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6301e7-3c5c-4492-a987-b526e1d1289d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:15.849922 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6301e7-3c5c-4492-a987-b526e1d1289d admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.850200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6301e7-3c5c-4492-a987-b526e1d1289d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.850516 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6301e7-3c5c-4492-a987-b526e1d1289d admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.851148 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6301e7-3c5c-4492-a987-b526e1d1289d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:15.853230 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6301e7-3c5c-4492-a987-b526e1d1289d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:15.875471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 964/1926] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:15 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:15.880002 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4afbe-6aa2-4ebd-8407-cd4df4b83ab7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:15.886384 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 963/1927] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:15 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:15.889731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6301e7-3c5c-4492-a987-b526e1d1289d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:15.894279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc4afbe-6aa2-4ebd-8407-cd4df4b83ab7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:15.920198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:15.943624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:15.957396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45f7e5-7e26-49b4-a00c-ed81b2636f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:15.958002 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45f7e5-7e26-49b4-a00c-ed81b2636f75 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.958305 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45f7e5-7e26-49b4-a00c-ed81b2636f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.958578 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af45f7e5-7e26-49b4-a00c-ed81b2636f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.959359 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45f7e5-7e26-49b4-a00c-ed81b2636f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:15.961282 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af45f7e5-7e26-49b4-a00c-ed81b2636f75 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:15.961502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6301e7-3c5c-4492-a987-b526e1d1289d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:15.966537 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1ca9d-7493-4b81-9c01-9aa391d6aabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:15.967206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1ca9d-7493-4b81-9c01-9aa391d6aabf admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:15.967490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1ca9d-7493-4b81-9c01-9aa391d6aabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:15.967765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a1ca9d-7493-4b81-9c01-9aa391d6aabf admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:15.968397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1ca9d-7493-4b81-9c01-9aa391d6aabf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:15.970547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a1ca9d-7493-4b81-9c01-9aa391d6aabf admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:15.993293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 965/1928] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:15 2022] POST /identity/v3/projects => generated 385 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:15.993969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 964/1929] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:15 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:15.997352 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45f7e5-7e26-49b4-a00c-ed81b2636f75 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:15.997859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1ca9d-7493-4b81-9c01-9aa391d6aabf admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.008698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a1ca9d-7493-4b81-9c01-9aa391d6aabf admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:16.066833 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af45f7e5-7e26-49b4-a00c-ed81b2636f75 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:16.071609 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e07840-0300-43b1-94e7-cb5c38689e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:16.072246 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e07840-0300-43b1-94e7-cb5c38689e34 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.072421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aee62-b01e-4c9f-b88e-79bd6145c955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:16.072533 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e07840-0300-43b1-94e7-cb5c38689e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.072788 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e07840-0300-43b1-94e7-cb5c38689e34 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.073147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aee62-b01e-4c9f-b88e-79bd6145c955 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.073415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e07840-0300-43b1-94e7-cb5c38689e34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:16.073513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aee62-b01e-4c9f-b88e-79bd6145c955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.073696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5aee62-b01e-4c9f-b88e-79bd6145c955 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.074403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5aee62-b01e-4c9f-b88e-79bd6145c955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:16.075391 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e07840-0300-43b1-94e7-cb5c38689e34 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:16.076028 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5aee62-b01e-4c9f-b88e-79bd6145c955 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:16.086957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 965/1930] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:16.090936 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aee62-b01e-4c9f-b88e-79bd6145c955 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.100999 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 966/1931] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:15 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:16.101588 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5aee62-b01e-4c9f-b88e-79bd6145c955 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:16.102071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e07840-0300-43b1-94e7-cb5c38689e34 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.106251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb599f-7089-4213-8cc8-8754b9fc9119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:16.106868 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb599f-7089-4213-8cc8-8754b9fc9119 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.107150 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb599f-7089-4213-8cc8-8754b9fc9119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.107421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edb599f-7089-4213-8cc8-8754b9fc9119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.108093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb599f-7089-4213-8cc8-8754b9fc9119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:16.109695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edb599f-7089-4213-8cc8-8754b9fc9119 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:16.112032 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e07840-0300-43b1-94e7-cb5c38689e34 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:16.116658 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e50e25-38a8-4d91-b899-0aa2a4b607d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:16.117263 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e50e25-38a8-4d91-b899-0aa2a4b607d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.117523 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e50e25-38a8-4d91-b899-0aa2a4b607d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.117787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e50e25-38a8-4d91-b899-0aa2a4b607d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.118473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e50e25-38a8-4d91-b899-0aa2a4b607d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:16.120091 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e50e25-38a8-4d91-b899-0aa2a4b607d8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:16.123667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 966/1932] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:16.124888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb599f-7089-4213-8cc8-8754b9fc9119 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.134105 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 967/1933] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:16.134729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edb599f-7089-4213-8cc8-8754b9fc9119 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:16.136128 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e50e25-38a8-4d91-b899-0aa2a4b607d8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.139925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4072f0-eb8d-4a7e-baca-17def1bb80ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:16.140562 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4072f0-eb8d-4a7e-baca-17def1bb80ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.140841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4072f0-eb8d-4a7e-baca-17def1bb80ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.141115 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4072f0-eb8d-4a7e-baca-17def1bb80ac admin admin] PATH_INFO: `/v3/projects/dbdec56ef7f04398abaca0511e5ed300/users/35077a1f2ac848deaa487bddba4eb763/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.146143 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e50e25-38a8-4d91-b899-0aa2a4b607d8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:16.150788 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1d839-c003-40d3-b3ee-90bd7d773a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:16.151440 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1d839-c003-40d3-b3ee-90bd7d773a85 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.151717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1d839-c003-40d3-b3ee-90bd7d773a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.151983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce1d839-c003-40d3-b3ee-90bd7d773a85 admin admin] PATH_INFO: `/v3/projects/956518baa58c414182341449a69de810/users/11da31f65ee64c3bb4fab478c0956999/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.158557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4072f0-eb8d-4a7e-baca-17def1bb80ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbdec56ef7f04398abaca0511e5ed300, user_id=35077a1f2ac848deaa487bddba4eb763, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:16.161219 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4072f0-eb8d-4a7e-baca-17def1bb80ac admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:16.168841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1d839-c003-40d3-b3ee-90bd7d773a85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=956518baa58c414182341449a69de810, user_id=11da31f65ee64c3bb4fab478c0956999, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:16.172086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce1d839-c003-40d3-b3ee-90bd7d773a85 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:16.175550 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 967/1934] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:16 2022] PUT /identity/v3/projects/dbdec56ef7f04398abaca0511e5ed300/users/35077a1f2ac848deaa487bddba4eb763/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:16.187642 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffc6ee-ee27-4cbe-b4ef-be52175e0797 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.187988 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffc6ee-ee27-4cbe-b4ef-be52175e0797 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.188269 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffc6ee-ee27-4cbe-b4ef-be52175e0797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.193713 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 968/1935] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:16 2022] PUT /identity/v3/projects/956518baa58c414182341449a69de810/users/11da31f65ee64c3bb4fab478c0956999/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:16.199137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1d839-c003-40d3-b3ee-90bd7d773a85 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.216871 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f0ffc6ee-ee27-4cbe-b4ef-be52175e0797 None None] MFA Rules not processed for user `35077a1f2ac848deaa487bddba4eb763`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:16.218245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f0ffc6ee-ee27-4cbe-b4ef-be52175e0797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:16.292427 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce1d839-c003-40d3-b3ee-90bd7d773a85 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:16.298506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 968/1936] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:16 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:16.298648 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1e426-a489-4911-9e2a-874a0fd30c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:16.299236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1e426-a489-4911-9e2a-874a0fd30c8d admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.299470 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffc6ee-ee27-4cbe-b4ef-be52175e0797 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.299585 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1e426-a489-4911-9e2a-874a0fd30c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.299884 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a1e426-a489-4911-9e2a-874a0fd30c8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.300553 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1e426-a489-4911-9e2a-874a0fd30c8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:16.302333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a1e426-a489-4911-9e2a-874a0fd30c8d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:16.313664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 969/1937] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:16.317987 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffc6ee-ee27-4cbe-b4ef-be52175e0797 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:16.318162 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1e426-a489-4911-9e2a-874a0fd30c8d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.323342 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:16.324284 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654581c-9c86-4a48-a866-b9b808b90eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:16.325658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654581c-9c86-4a48-a866-b9b808b90eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.325876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654581c-9c86-4a48-a866-b9b808b90eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.326380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c654581c-9c86-4a48-a866-b9b808b90eb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.327986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654581c-9c86-4a48-a866-b9b808b90eb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:16.329241 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a1e426-a489-4911-9e2a-874a0fd30c8d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:16.329900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c654581c-9c86-4a48-a866-b9b808b90eb9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:16.338023 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa3996-4b67-4f05-8e8d-3519ad308a29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:16.338754 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa3996-4b67-4f05-8e8d-3519ad308a29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.339038 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa3996-4b67-4f05-8e8d-3519ad308a29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.339318 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa3996-4b67-4f05-8e8d-3519ad308a29 admin admin] PATH_INFO: `/v3/projects/956518baa58c414182341449a69de810/users/11da31f65ee64c3bb4fab478c0956999/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.343107 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa3996-4b67-4f05-8e8d-3519ad308a29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=956518baa58c414182341449a69de810, user_id=11da31f65ee64c3bb4fab478c0956999, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:16.346125 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa3996-4b67-4f05-8e8d-3519ad308a29 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:16.368658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 969/1938] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:16 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:16.371935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 970/1939] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:16 2022] PUT /identity/v3/projects/956518baa58c414182341449a69de810/users/11da31f65ee64c3bb4fab478c0956999/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:16.375584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654581c-9c86-4a48-a866-b9b808b90eb9 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.384571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355548d9-3ad8-4c5c-aa73-bdd5390b7f2b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.384927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355548d9-3ad8-4c5c-aa73-bdd5390b7f2b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.385145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355548d9-3ad8-4c5c-aa73-bdd5390b7f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.394776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:16.434583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-355548d9-3ad8-4c5c-aa73-bdd5390b7f2b None None] MFA Rules not processed for user `11da31f65ee64c3bb4fab478c0956999`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:16.437278 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-355548d9-3ad8-4c5c-aa73-bdd5390b7f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:16.537126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c654581c-9c86-4a48-a866-b9b808b90eb9 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:16.542087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0418a8-234d-4240-b3a3-0f487714f183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:16.542730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0418a8-234d-4240-b3a3-0f487714f183 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.543015 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0418a8-234d-4240-b3a3-0f487714f183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.543286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0418a8-234d-4240-b3a3-0f487714f183 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.543913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0418a8-234d-4240-b3a3-0f487714f183 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:16.546389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0418a8-234d-4240-b3a3-0f487714f183 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:16.555649 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 971/1940] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:16 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:16.568902 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355548d9-3ad8-4c5c-aa73-bdd5390b7f2b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.570409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-355548d9-3ad8-4c5c-aa73-bdd5390b7f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:16.575388 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 970/1941] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:16 2022] POST /identity/v3/users => generated 295 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:16.576422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0418a8-234d-4240-b3a3-0f487714f183 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.586408 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0418a8-234d-4240-b3a3-0f487714f183 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:16.591086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437286ba-0a3c-446e-a48f-728e0ab0f514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:16.591673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437286ba-0a3c-446e-a48f-728e0ab0f514 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.591949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437286ba-0a3c-446e-a48f-728e0ab0f514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.592218 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437286ba-0a3c-446e-a48f-728e0ab0f514 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.592843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437286ba-0a3c-446e-a48f-728e0ab0f514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:16.594729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437286ba-0a3c-446e-a48f-728e0ab0f514 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:16.607624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 971/1942] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:16.609699 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437286ba-0a3c-446e-a48f-728e0ab0f514 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.621101 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437286ba-0a3c-446e-a48f-728e0ab0f514 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:16.626501 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abaa49e-2bdc-4277-854e-3e256604e703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:16.627255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abaa49e-2bdc-4277-854e-3e256604e703 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.627540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abaa49e-2bdc-4277-854e-3e256604e703 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.627818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abaa49e-2bdc-4277-854e-3e256604e703 admin admin] PATH_INFO: `/v3/projects/d6321ab627154e5d894f64c1aae7daa2/users/2ec90633aa1341f6932ca355f2ecfb84/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.632321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355548d9-3ad8-4c5c-aa73-bdd5390b7f2b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:16.636049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c7696-1fa0-464d-ab5e-f1f7490495b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:16.636648 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c7696-1fa0-464d-ab5e-f1f7490495b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.636921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c7696-1fa0-464d-ab5e-f1f7490495b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.637188 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604c7696-1fa0-464d-ab5e-f1f7490495b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.644278 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abaa49e-2bdc-4277-854e-3e256604e703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6321ab627154e5d894f64c1aae7daa2, user_id=2ec90633aa1341f6932ca355f2ecfb84, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:16.646899 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c7696-1fa0-464d-ab5e-f1f7490495b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:16.647047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abaa49e-2bdc-4277-854e-3e256604e703 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:16.648321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604c7696-1fa0-464d-ab5e-f1f7490495b3 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:16.658703 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 972/1943] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:16.662730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 972/1944] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:16 2022] PUT /identity/v3/projects/d6321ab627154e5d894f64c1aae7daa2/users/2ec90633aa1341f6932ca355f2ecfb84/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:16.670246 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c7696-1fa0-464d-ab5e-f1f7490495b3 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.689052 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604c7696-1fa0-464d-ab5e-f1f7490495b3 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:16.689204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abaa49e-2bdc-4277-854e-3e256604e703 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.746872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f28915-a20f-47f3-99e2-22cb07fb9d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:16.747488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f28915-a20f-47f3-99e2-22cb07fb9d0a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.747765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f28915-a20f-47f3-99e2-22cb07fb9d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.748038 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f28915-a20f-47f3-99e2-22cb07fb9d0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.749229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f28915-a20f-47f3-99e2-22cb07fb9d0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:16.750867 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f28915-a20f-47f3-99e2-22cb07fb9d0a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:16.752550 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abaa49e-2bdc-4277-854e-3e256604e703 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:16.757312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9b697-3cf6-4613-8d40-d8c13ab81771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:16.757945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9b697-3cf6-4613-8d40-d8c13ab81771 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.758249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9b697-3cf6-4613-8d40-d8c13ab81771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.758522 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9b697-3cf6-4613-8d40-d8c13ab81771 admin admin] PATH_INFO: `/v3/users/63558284bd8242a8b149b44ffb7d34fd` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.759648 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9b697-3cf6-4613-8d40-d8c13ab81771 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63558284bd8242a8b149b44ffb7d34fd)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:16.761015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 973/1945] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:16 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:16.764455 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9b697-3cf6-4613-8d40-d8c13ab81771 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:16.765141 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f28915-a20f-47f3-99e2-22cb07fb9d0a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.774717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f28915-a20f-47f3-99e2-22cb07fb9d0a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:16.779469 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4320087b-5832-4d00-b7ba-81e5194a39c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:16.780116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320087b-5832-4d00-b7ba-81e5194a39c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.780415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320087b-5832-4d00-b7ba-81e5194a39c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.780685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320087b-5832-4d00-b7ba-81e5194a39c1 admin admin] PATH_INFO: `/v3/projects/d6321ab627154e5d894f64c1aae7daa2/users/2ec90633aa1341f6932ca355f2ecfb84/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.783162 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320087b-5832-4d00-b7ba-81e5194a39c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6321ab627154e5d894f64c1aae7daa2, user_id=2ec90633aa1341f6932ca355f2ecfb84, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:16.785487 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4320087b-5832-4d00-b7ba-81e5194a39c1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:16.792555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:16.802826 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 974/1946] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:16 2022] PUT /identity/v3/projects/d6321ab627154e5d894f64c1aae7daa2/users/2ec90633aa1341f6932ca355f2ecfb84/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:16.812312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed6a71-1ba5-40f6-91cf-73d6e8e1d00c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.813186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed6a71-1ba5-40f6-91cf-73d6e8e1d00c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.813619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed6a71-1ba5-40f6-91cf-73d6e8e1d00c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.819995 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-68f9b697-3cf6-4613-8d40-d8c13ab81771 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63558284bd8242a8b149b44ffb7d34fd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:16.829469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-68f9b697-3cf6-4613-8d40-d8c13ab81771 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63558284bd8242a8b149b44ffb7d34fd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:16.839883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:16.847222 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3fed6a71-1ba5-40f6-91cf-73d6e8e1d00c None None] MFA Rules not processed for user `2ec90633aa1341f6932ca355f2ecfb84`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:16.847465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-68f9b697-3cf6-4613-8d40-d8c13ab81771 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63558284bd8242a8b149b44ffb7d34fd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:16.848058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-68f9b697-3cf6-4613-8d40-d8c13ab81771 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63558284bd8242a8b149b44ffb7d34fd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:16.848805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3fed6a71-1ba5-40f6-91cf-73d6e8e1d00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:16.861092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-68f9b697-3cf6-4613-8d40-d8c13ab81771 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63558284bd8242a8b149b44ffb7d34fd'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:16.868612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 973/1947] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:16 2022] DELETE /identity/v3/users/63558284bd8242a8b149b44ffb7d34fd => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:16.873459 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07213721-b25f-497c-afc8-6065ae43b042 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:16.874398 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07213721-b25f-497c-afc8-6065ae43b042 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:16.874690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07213721-b25f-497c-afc8-6065ae43b042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:16.895716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-07213721-b25f-497c-afc8-6065ae43b042 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:16.897416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-07213721-b25f-497c-afc8-6065ae43b042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:16.941886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 975/1948] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:16 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:16.951561 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed6a71-1ba5-40f6-91cf-73d6e8e1d00c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.953146 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3fed6a71-1ba5-40f6-91cf-73d6e8e1d00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:16.970307 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 974/1949] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:16.987892 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07213721-b25f-497c-afc8-6065ae43b042 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:16.989408 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-07213721-b25f-497c-afc8-6065ae43b042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:17.025133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed6a71-1ba5-40f6-91cf-73d6e8e1d00c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:17.054106 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07213721-b25f-497c-afc8-6065ae43b042 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:17.058726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16ba6c-e575-4766-bf74-0ad1b84272c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:17.059340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16ba6c-e575-4766-bf74-0ad1b84272c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:17.059620 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16ba6c-e575-4766-bf74-0ad1b84272c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:17.059888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d16ba6c-e575-4766-bf74-0ad1b84272c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:17.071353 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16ba6c-e575-4766-bf74-0ad1b84272c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:17.072803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d16ba6c-e575-4766-bf74-0ad1b84272c0 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:17.089304 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 975/1950] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:17.095310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e4178-18ef-4585-8694-932dd5514cb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:17.096064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e4178-18ef-4585-8694-932dd5514cb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:17.096489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e4178-18ef-4585-8694-932dd5514cb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:17.096968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394e4178-18ef-4585-8694-932dd5514cb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:17.097146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16ba6c-e575-4766-bf74-0ad1b84272c0 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:17.099087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8d16ba6c-e575-4766-bf74-0ad1b84272c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:17.110975 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d16ba6c-e575-4766-bf74-0ad1b84272c0 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:17.114775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f851899-89c6-49b8-a7b0-25dd7e96b685 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:17.115369 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f851899-89c6-49b8-a7b0-25dd7e96b685 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:17.115646 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f851899-89c6-49b8-a7b0-25dd7e96b685 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:17.115914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f851899-89c6-49b8-a7b0-25dd7e96b685 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:17.117653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5f851899-89c6-49b8-a7b0-25dd7e96b685 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:17.164577 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:17.177760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e4178-18ef-4585-8694-932dd5514cb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:17.179358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394e4178-18ef-4585-8694-932dd5514cb6 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:17.190735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 976/1951] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:16 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:17.199159 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f851899-89c6-49b8-a7b0-25dd7e96b685 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:17.200844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f851899-89c6-49b8-a7b0-25dd7e96b685 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:17.214806 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 976/1952] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:17 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:17.234167 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e4178-18ef-4585-8694-932dd5514cb6 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:17.243758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394e4178-18ef-4585-8694-932dd5514cb6 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:17.263817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdbf71-8386-446e-91bd-4135d2e57ee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:17.265466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdbf71-8386-446e-91bd-4135d2e57ee0 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:17.265806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdbf71-8386-446e-91bd-4135d2e57ee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:17.266124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cdbf71-8386-446e-91bd-4135d2e57ee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:17.268232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a7cdbf71-8386-446e-91bd-4135d2e57ee0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:17.348961 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdbf71-8386-446e-91bd-4135d2e57ee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:17.350550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cdbf71-8386-446e-91bd-4135d2e57ee0 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:17.361299 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 977/1953] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:17 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:17.417829 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f851899-89c6-49b8-a7b0-25dd7e96b685 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:17.419416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5f851899-89c6-49b8-a7b0-25dd7e96b685 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:17.440030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f851899-89c6-49b8-a7b0-25dd7e96b685 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:17.446300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd97c83-244e-46a5-921b-8d17fea002f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:17.447343 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd97c83-244e-46a5-921b-8d17fea002f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:17.447667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd97c83-244e-46a5-921b-8d17fea002f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:17.447985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd97c83-244e-46a5-921b-8d17fea002f3 admin admin] PATH_INFO: `/v3/projects/b2e760e99a364299b9b3af03d6a2addd` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:17.449589 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd97c83-244e-46a5-921b-8d17fea002f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2e760e99a364299b9b3af03d6a2addd)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:17.452160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd97c83-244e-46a5-921b-8d17fea002f3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:17.466684 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:17.537253 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8fd97c83-244e-46a5-921b-8d17fea002f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2e760e99a364299b9b3af03d6a2addd', 'request_id': 'req-8fd97c83-244e-46a5-921b-8d17fea002f3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:17.545778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8fd97c83-244e-46a5-921b-8d17fea002f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2e760e99a364299b9b3af03d6a2addd', 'request_id': 'req-8fd97c83-244e-46a5-921b-8d17fea002f3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:17.558120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8fd97c83-244e-46a5-921b-8d17fea002f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2e760e99a364299b9b3af03d6a2addd', 'request_id': 'req-8fd97c83-244e-46a5-921b-8d17fea002f3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:17.586611 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8fd97c83-244e-46a5-921b-8d17fea002f3 admin admin] The token cache is being invalidate because project b2e760e99a364299b9b3af03d6a2addd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:17.587165 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8fd97c83-244e-46a5-921b-8d17fea002f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:17.588665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8fd97c83-244e-46a5-921b-8d17fea002f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:17.591552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 977/1954] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:17 2022] DELETE /identity/v3/projects/b2e760e99a364299b9b3af03d6a2addd => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:17.595949 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdbf71-8386-446e-91bd-4135d2e57ee0 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:17.631604 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cdbf71-8386-446e-91bd-4135d2e57ee0 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:17.733267 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a25343-3884-4c32-b230-8e5e7fa6ad96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:17.734921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a25343-3884-4c32-b230-8e5e7fa6ad96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:17.735253 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a25343-3884-4c32-b230-8e5e7fa6ad96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:17.735502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a25343-3884-4c32-b230-8e5e7fa6ad96 admin admin] PATH_INFO: `/v3/users/11da31f65ee64c3bb4fab478c0956999` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:17.750250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a25343-3884-4c32-b230-8e5e7fa6ad96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11da31f65ee64c3bb4fab478c0956999)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:17.750250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a25343-3884-4c32-b230-8e5e7fa6ad96 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:17.827865 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d9a25343-3884-4c32-b230-8e5e7fa6ad96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11da31f65ee64c3bb4fab478c0956999'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:17.828602 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d9a25343-3884-4c32-b230-8e5e7fa6ad96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11da31f65ee64c3bb4fab478c0956999'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:17.857624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d9a25343-3884-4c32-b230-8e5e7fa6ad96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11da31f65ee64c3bb4fab478c0956999'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:17.869860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d9a25343-3884-4c32-b230-8e5e7fa6ad96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11da31f65ee64c3bb4fab478c0956999'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:17.870810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d9a25343-3884-4c32-b230-8e5e7fa6ad96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11da31f65ee64c3bb4fab478c0956999'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:17.886361 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 978/1955] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:17 2022] DELETE /identity/v3/users/11da31f65ee64c3bb4fab478c0956999 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:18.178965 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd97c83-244e-46a5-921b-8d17fea002f3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:18.180435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8fd97c83-244e-46a5-921b-8d17fea002f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:18.197760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd97c83-244e-46a5-921b-8d17fea002f3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:18.255509 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88594d4e-7022-4a69-8ae0-224163dbc3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:18.256140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88594d4e-7022-4a69-8ae0-224163dbc3ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:18.256625 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88594d4e-7022-4a69-8ae0-224163dbc3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:18.257389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88594d4e-7022-4a69-8ae0-224163dbc3ef admin admin] PATH_INFO: `/v3/projects/956518baa58c414182341449a69de810` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:18.259172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88594d4e-7022-4a69-8ae0-224163dbc3ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=956518baa58c414182341449a69de810)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:18.261079 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88594d4e-7022-4a69-8ae0-224163dbc3ef admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:18.308211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-88594d4e-7022-4a69-8ae0-224163dbc3ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '956518baa58c414182341449a69de810', 'request_id': 'req-88594d4e-7022-4a69-8ae0-224163dbc3ef'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:18.314837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-88594d4e-7022-4a69-8ae0-224163dbc3ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '956518baa58c414182341449a69de810', 'request_id': 'req-88594d4e-7022-4a69-8ae0-224163dbc3ef'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:18.329665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-88594d4e-7022-4a69-8ae0-224163dbc3ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '956518baa58c414182341449a69de810', 'request_id': 'req-88594d4e-7022-4a69-8ae0-224163dbc3ef'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:18.341870 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-88594d4e-7022-4a69-8ae0-224163dbc3ef admin admin] The token cache is being invalidate because project 956518baa58c414182341449a69de810 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:18.342451 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-88594d4e-7022-4a69-8ae0-224163dbc3ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:18.343099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-88594d4e-7022-4a69-8ae0-224163dbc3ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:18.344589 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 978/1956] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:18 2022] DELETE /identity/v3/projects/956518baa58c414182341449a69de810 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:18.349434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5467e9-2400-417f-bc2b-5c43e6157df9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:18.349920 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5467e9-2400-417f-bc2b-5c43e6157df9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:18.350007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5467e9-2400-417f-bc2b-5c43e6157df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:18.369110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9a5467e9-2400-417f-bc2b-5c43e6157df9 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:18.370353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9a5467e9-2400-417f-bc2b-5c43e6157df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:18.375479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:18.435386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 979/1957] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:18.443447 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23259262-a543-47b9-861f-985f823283c7 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:18.443758 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23259262-a543-47b9-861f-985f823283c7 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:18.444034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23259262-a543-47b9-861f-985f823283c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:18.463521 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-23259262-a543-47b9-861f-985f823283c7 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:18.464627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-23259262-a543-47b9-861f-985f823283c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:18.473328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 979/1958] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:18.477558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5467e9-2400-417f-bc2b-5c43e6157df9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:18.487970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5467e9-2400-417f-bc2b-5c43e6157df9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:18.492444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa08350-0db0-4576-94af-553f9e2220af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:18.493033 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa08350-0db0-4576-94af-553f9e2220af admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:18.493305 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa08350-0db0-4576-94af-553f9e2220af admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:18.493572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa08350-0db0-4576-94af-553f9e2220af admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:18.494203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa08350-0db0-4576-94af-553f9e2220af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:18.494522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa08350-0db0-4576-94af-553f9e2220af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:18.496375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa08350-0db0-4576-94af-553f9e2220af admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:18.509699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 980/1959] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:27:18.514504 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23259262-a543-47b9-861f-985f823283c7 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:18.524319 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23259262-a543-47b9-861f-985f823283c7 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:18.528953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115c457-34bf-4aed-b433-37646606464c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:18.529586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115c457-34bf-4aed-b433-37646606464c admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:18.530454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115c457-34bf-4aed-b433-37646606464c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:18.530730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115c457-34bf-4aed-b433-37646606464c admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:18.531374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115c457-34bf-4aed-b433-37646606464c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:18.533547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115c457-34bf-4aed-b433-37646606464c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:18.561038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 980/1960] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:18 2022] POST /identity/v3/projects => generated 381 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:18.565397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa08350-0db0-4576-94af-553f9e2220af admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:18.575427 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa08350-0db0-4576-94af-553f9e2220af admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:18.632309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b5107-86d0-44d3-a7de-78ee78966c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:18.632925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b5107-86d0-44d3-a7de-78ee78966c92 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:18.633205 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b5107-86d0-44d3-a7de-78ee78966c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:18.633592 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3b5107-86d0-44d3-a7de-78ee78966c92 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:18.634100 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b5107-86d0-44d3-a7de-78ee78966c92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:18.636041 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3b5107-86d0-44d3-a7de-78ee78966c92 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:18.662656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 981/1961] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:18 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:18.663840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115c457-34bf-4aed-b433-37646606464c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:18.669627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:18.676391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115c457-34bf-4aed-b433-37646606464c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:18.684720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b6191-a685-4944-88ac-af6bfc48361d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:18.685671 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b6191-a685-4944-88ac-af6bfc48361d admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:18.686601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b6191-a685-4944-88ac-af6bfc48361d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:18.686926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19b6191-a685-4944-88ac-af6bfc48361d admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:18.687868 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b6191-a685-4944-88ac-af6bfc48361d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:18.690175 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19b6191-a685-4944-88ac-af6bfc48361d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:18.708902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 981/1962] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:18.712922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b5107-86d0-44d3-a7de-78ee78966c92 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:18.723873 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3b5107-86d0-44d3-a7de-78ee78966c92 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:18.728427 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5db870-873c-41ac-a2f1-05ab955385bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:18.729022 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5db870-873c-41ac-a2f1-05ab955385bd admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:18.729296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5db870-873c-41ac-a2f1-05ab955385bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:18.729568 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5db870-873c-41ac-a2f1-05ab955385bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:18.730205 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5db870-873c-41ac-a2f1-05ab955385bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:18.731821 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5db870-873c-41ac-a2f1-05ab955385bd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:18.742246 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 982/1963] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:18.746422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b6191-a685-4944-88ac-af6bfc48361d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:18.756687 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19b6191-a685-4944-88ac-af6bfc48361d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:18.761386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57f27f-74a6-4d6d-83d6-139709d2a7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:18.762043 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57f27f-74a6-4d6d-83d6-139709d2a7c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:18.762372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57f27f-74a6-4d6d-83d6-139709d2a7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:18.762648 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e57f27f-74a6-4d6d-83d6-139709d2a7c3 admin admin] PATH_INFO: `/v3/projects/c952d6fe2942408a9e6a4405f6e565dd/users/50d35ee594df41c8a1bd329efca47935/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:18.780287 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e57f27f-74a6-4d6d-83d6-139709d2a7c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c952d6fe2942408a9e6a4405f6e565dd, user_id=50d35ee594df41c8a1bd329efca47935, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:18.783259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e57f27f-74a6-4d6d-83d6-139709d2a7c3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:18.799366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 982/1964] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:18 2022] PUT /identity/v3/projects/c952d6fe2942408a9e6a4405f6e565dd/users/50d35ee594df41c8a1bd329efca47935/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:18.810742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dee9db-6904-4c93-8140-2d78d3eb387f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:18.811104 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dee9db-6904-4c93-8140-2d78d3eb387f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:18.811326 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dee9db-6904-4c93-8140-2d78d3eb387f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:18.838995 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-90dee9db-6904-4c93-8140-2d78d3eb387f None None] MFA Rules not processed for user `50d35ee594df41c8a1bd329efca47935`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:18.840207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-90dee9db-6904-4c93-8140-2d78d3eb387f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:18.934293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 983/1965] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:18 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:18.939154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57f27f-74a6-4d6d-83d6-139709d2a7c3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:19.008507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e57f27f-74a6-4d6d-83d6-139709d2a7c3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:19.013187 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e5bff-c950-42ab-8837-d3d05ba0632b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:19.013859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e5bff-c950-42ab-8837-d3d05ba0632b admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:19.014905 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e5bff-c950-42ab-8837-d3d05ba0632b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:19.015188 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260e5bff-c950-42ab-8837-d3d05ba0632b admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:19.015812 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e5bff-c950-42ab-8837-d3d05ba0632b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:19.017702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260e5bff-c950-42ab-8837-d3d05ba0632b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:19.045984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 983/1966] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:18 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:19.050571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dee9db-6904-4c93-8140-2d78d3eb387f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:19.057249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:19.072668 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dee9db-6904-4c93-8140-2d78d3eb387f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:19.144401 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d2870-51cf-466b-83ee-9a5616bfbb10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:19.145000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d2870-51cf-466b-83ee-9a5616bfbb10 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:19.145279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d2870-51cf-466b-83ee-9a5616bfbb10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:19.145552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16d2870-51cf-466b-83ee-9a5616bfbb10 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:19.146185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d2870-51cf-466b-83ee-9a5616bfbb10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:19.148155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16d2870-51cf-466b-83ee-9a5616bfbb10 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:19.171849 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 984/1967] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:19 2022] POST /identity/v3/users => generated 298 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:19.176109 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e5bff-c950-42ab-8837-d3d05ba0632b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:19.187911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260e5bff-c950-42ab-8837-d3d05ba0632b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:19.193406 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6012c34-745a-46f0-8f3b-a2c624adff3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:19.194109 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6012c34-745a-46f0-8f3b-a2c624adff3a admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:19.194574 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6012c34-745a-46f0-8f3b-a2c624adff3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:19.194845 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6012c34-745a-46f0-8f3b-a2c624adff3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:19.195697 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6012c34-745a-46f0-8f3b-a2c624adff3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:19.197488 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6012c34-745a-46f0-8f3b-a2c624adff3a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:19.209944 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 984/1968] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:19.214367 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d2870-51cf-466b-83ee-9a5616bfbb10 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:19.224834 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16d2870-51cf-466b-83ee-9a5616bfbb10 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:19.229453 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540ece3-fa2e-4c54-b324-152b12ffcad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:19.230102 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540ece3-fa2e-4c54-b324-152b12ffcad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:19.230412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540ece3-fa2e-4c54-b324-152b12ffcad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:19.230689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1540ece3-fa2e-4c54-b324-152b12ffcad4 admin admin] PATH_INFO: `/v3/projects/fde17b6ad5944510950ce7bc2f034071/users/2c0cdba7439b40bbb649da3466beaf52/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:19.245686 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540ece3-fa2e-4c54-b324-152b12ffcad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fde17b6ad5944510950ce7bc2f034071, user_id=2c0cdba7439b40bbb649da3466beaf52, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:19.248477 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1540ece3-fa2e-4c54-b324-152b12ffcad4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:19.262941 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 985/1969] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:19 2022] PUT /identity/v3/projects/fde17b6ad5944510950ce7bc2f034071/users/2c0cdba7439b40bbb649da3466beaf52/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:19.267558 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6012c34-745a-46f0-8f3b-a2c624adff3a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:19.278753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6012c34-745a-46f0-8f3b-a2c624adff3a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:19.340082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18549012-1765-4804-bd04-ca4380b09a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:19.340767 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18549012-1765-4804-bd04-ca4380b09a2e admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:19.341081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18549012-1765-4804-bd04-ca4380b09a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:19.341382 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18549012-1765-4804-bd04-ca4380b09a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:19.342343 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18549012-1765-4804-bd04-ca4380b09a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:19.344816 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18549012-1765-4804-bd04-ca4380b09a2e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:19.356742 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 985/1970] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:19 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:19.360987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540ece3-fa2e-4c54-b324-152b12ffcad4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:19.371602 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1540ece3-fa2e-4c54-b324-152b12ffcad4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:19.376273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a7529-1b2b-4d28-ba58-fcd6c4014045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:19.377023 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a7529-1b2b-4d28-ba58-fcd6c4014045 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:19.377195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a7529-1b2b-4d28-ba58-fcd6c4014045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:19.377462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a7529-1b2b-4d28-ba58-fcd6c4014045 admin admin] PATH_INFO: `/v3/projects/fde17b6ad5944510950ce7bc2f034071/users/2c0cdba7439b40bbb649da3466beaf52/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:19.379419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a7529-1b2b-4d28-ba58-fcd6c4014045 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fde17b6ad5944510950ce7bc2f034071, user_id=2c0cdba7439b40bbb649da3466beaf52, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:19.382067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44a7529-1b2b-4d28-ba58-fcd6c4014045 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:19.395473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 986/1971] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:19 2022] PUT /identity/v3/projects/fde17b6ad5944510950ce7bc2f034071/users/2c0cdba7439b40bbb649da3466beaf52/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:19.407800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede142ba-98e5-4720-ae02-0164b795b4ac None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:19.408250 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede142ba-98e5-4720-ae02-0164b795b4ac None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:19.408635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede142ba-98e5-4720-ae02-0164b795b4ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:19.446760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ede142ba-98e5-4720-ae02-0164b795b4ac None None] MFA Rules not processed for user `2c0cdba7439b40bbb649da3466beaf52`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:19.447987 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ede142ba-98e5-4720-ae02-0164b795b4ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:19.475570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:19.523720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 986/1972] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:19.536185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a7529-1b2b-4d28-ba58-fcd6c4014045 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:19.537949 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a44a7529-1b2b-4d28-ba58-fcd6c4014045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:19.597737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44a7529-1b2b-4d28-ba58-fcd6c4014045 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:19.601530 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d1e62-2f99-4cce-a87b-a45c89d42e9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:19.602136 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d1e62-2f99-4cce-a87b-a45c89d42e9b service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:19.602445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d1e62-2f99-4cce-a87b-a45c89d42e9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:19.602716 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611d1e62-2f99-4cce-a87b-a45c89d42e9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:19.611523 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d1e62-2f99-4cce-a87b-a45c89d42e9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:19.612944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611d1e62-2f99-4cce-a87b-a45c89d42e9b service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:19.622366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 987/1973] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:19.712718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede142ba-98e5-4720-ae02-0164b795b4ac None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:19.722774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede142ba-98e5-4720-ae02-0164b795b4ac None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:19.726402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0679d3d7-5b60-413b-81de-06cbe5118e06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:19.727006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679d3d7-5b60-413b-81de-06cbe5118e06 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:19.727313 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679d3d7-5b60-413b-81de-06cbe5118e06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:19.727580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0679d3d7-5b60-413b-81de-06cbe5118e06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:19.797682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0679d3d7-5b60-413b-81de-06cbe5118e06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:19.799323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0679d3d7-5b60-413b-81de-06cbe5118e06 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:19.808420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 987/1974] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:20.098653 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d1e62-2f99-4cce-a87b-a45c89d42e9b service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:20.165641 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611d1e62-2f99-4cce-a87b-a45c89d42e9b service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:20.170839 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465a118-066b-407f-9a85-3a417d103bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:20.171502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465a118-066b-407f-9a85-3a417d103bb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:20.172065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465a118-066b-407f-9a85-3a417d103bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:20.172243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465a118-066b-407f-9a85-3a417d103bb2 admin admin] PATH_INFO: `/v3/users/50d35ee594df41c8a1bd329efca47935` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:20.177763 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465a118-066b-407f-9a85-3a417d103bb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50d35ee594df41c8a1bd329efca47935)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:20.179759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465a118-066b-407f-9a85-3a417d103bb2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:20.181564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:20.231671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e465a118-066b-407f-9a85-3a417d103bb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50d35ee594df41c8a1bd329efca47935'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:20.232322 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e465a118-066b-407f-9a85-3a417d103bb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50d35ee594df41c8a1bd329efca47935'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:20.245686 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e465a118-066b-407f-9a85-3a417d103bb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50d35ee594df41c8a1bd329efca47935'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:20.254502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e465a118-066b-407f-9a85-3a417d103bb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50d35ee594df41c8a1bd329efca47935'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:20.255112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e465a118-066b-407f-9a85-3a417d103bb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50d35ee594df41c8a1bd329efca47935'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:20.268550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 988/1975] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:20 2022] DELETE /identity/v3/users/50d35ee594df41c8a1bd329efca47935 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:20.275655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a618458-2bdb-40f9-8cae-e749807f2ffd None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:20.276125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a618458-2bdb-40f9-8cae-e749807f2ffd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:20.276365 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a618458-2bdb-40f9-8cae-e749807f2ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:20.297578 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5a618458-2bdb-40f9-8cae-e749807f2ffd None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:20.298825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5a618458-2bdb-40f9-8cae-e749807f2ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:20.358109 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 988/1976] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:20.367856 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465a118-066b-407f-9a85-3a417d103bb2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:20.369302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e465a118-066b-407f-9a85-3a417d103bb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:20.427173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465a118-066b-407f-9a85-3a417d103bb2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:20.430917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b3655-f2d8-42fb-817a-b6050d11d1ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:20.431514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b3655-f2d8-42fb-817a-b6050d11d1ff service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:20.431789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b3655-f2d8-42fb-817a-b6050d11d1ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:20.432055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71b3655-f2d8-42fb-817a-b6050d11d1ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:20.441371 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b3655-f2d8-42fb-817a-b6050d11d1ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:20.443367 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71b3655-f2d8-42fb-817a-b6050d11d1ff service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:20.453279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 989/1977] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:20.755288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a618458-2bdb-40f9-8cae-e749807f2ffd None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:20.758522 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5a618458-2bdb-40f9-8cae-e749807f2ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:20.778090 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a618458-2bdb-40f9-8cae-e749807f2ffd None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:20.785138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c77abcc-b9fb-40e8-b8ec-f96a3b804c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:20.785921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77abcc-b9fb-40e8-b8ec-f96a3b804c7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:20.786844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77abcc-b9fb-40e8-b8ec-f96a3b804c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:20.787129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c77abcc-b9fb-40e8-b8ec-f96a3b804c7d admin admin] PATH_INFO: `/v3/projects/c952d6fe2942408a9e6a4405f6e565dd` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:20.788796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77abcc-b9fb-40e8-b8ec-f96a3b804c7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c952d6fe2942408a9e6a4405f6e565dd)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:20.791362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c77abcc-b9fb-40e8-b8ec-f96a3b804c7d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:20.869260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7c77abcc-b9fb-40e8-b8ec-f96a3b804c7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c952d6fe2942408a9e6a4405f6e565dd', 'request_id': 'req-7c77abcc-b9fb-40e8-b8ec-f96a3b804c7d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:20.878124 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7c77abcc-b9fb-40e8-b8ec-f96a3b804c7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c952d6fe2942408a9e6a4405f6e565dd', 'request_id': 'req-7c77abcc-b9fb-40e8-b8ec-f96a3b804c7d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:20.881588 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:20.890300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7c77abcc-b9fb-40e8-b8ec-f96a3b804c7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c952d6fe2942408a9e6a4405f6e565dd', 'request_id': 'req-7c77abcc-b9fb-40e8-b8ec-f96a3b804c7d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:20.912102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7c77abcc-b9fb-40e8-b8ec-f96a3b804c7d admin admin] The token cache is being invalidate because project c952d6fe2942408a9e6a4405f6e565dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:20.912405 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7c77abcc-b9fb-40e8-b8ec-f96a3b804c7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:20.913075 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7c77abcc-b9fb-40e8-b8ec-f96a3b804c7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:20.919684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b3655-f2d8-42fb-817a-b6050d11d1ff service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:20.929802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 989/1978] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:20 2022] DELETE /identity/v3/projects/c952d6fe2942408a9e6a4405f6e565dd => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:20.931321 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dd823-7ba4-4fab-8add-7eee03f112d0 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:20.931608 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dd823-7ba4-4fab-8add-7eee03f112d0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:20.931880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9dd823-7ba4-4fab-8add-7eee03f112d0 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:20.933400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 990/1979] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 15:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:27:20.935458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71b3655-f2d8-42fb-817a-b6050d11d1ff service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:20.945419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd1cad-c89e-4456-9dbb-e9538d5a213c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:20.945917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd1cad-c89e-4456-9dbb-e9538d5a213c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:20.946123 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd1cad-c89e-4456-9dbb-e9538d5a213c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:20.985481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4edd1cad-c89e-4456-9dbb-e9538d5a213c None None] MFA Rules not processed for user `8a5fd31a2db944f596912e4f0aa26835`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:20.988645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4edd1cad-c89e-4456-9dbb-e9538d5a213c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:21.091622 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 991/1980] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 15:27:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:21.108363 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd1cad-c89e-4456-9dbb-e9538d5a213c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:21.110160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4edd1cad-c89e-4456-9dbb-e9538d5a213c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:21.113736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b9821d-4fca-4eb0-97ac-896e9741eca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:21.114423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9821d-4fca-4eb0-97ac-896e9741eca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:21.114700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9821d-4fca-4eb0-97ac-896e9741eca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:21.114970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b9821d-4fca-4eb0-97ac-896e9741eca8 admin admin] PATH_INFO: `/v3/users/2c0cdba7439b40bbb649da3466beaf52` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:21.116092 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9821d-4fca-4eb0-97ac-896e9741eca8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c0cdba7439b40bbb649da3466beaf52)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:21.117984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b9821d-4fca-4eb0-97ac-896e9741eca8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:21.125248 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd1cad-c89e-4456-9dbb-e9538d5a213c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:21.137432 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1797e-19eb-4cfb-9fa8-8282b72f945e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:21.138060 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1797e-19eb-4cfb-9fa8-8282b72f945e service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:21.138388 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1797e-19eb-4cfb-9fa8-8282b72f945e service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:21.138835 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa1797e-19eb-4cfb-9fa8-8282b72f945e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:21.149096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1797e-19eb-4cfb-9fa8-8282b72f945e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:21.149641 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:21.150792 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa1797e-19eb-4cfb-9fa8-8282b72f945e service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:21.164599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 992/1981] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:21.191794 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f4b9821d-4fca-4eb0-97ac-896e9741eca8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c0cdba7439b40bbb649da3466beaf52'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:21.193053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f4b9821d-4fca-4eb0-97ac-896e9741eca8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c0cdba7439b40bbb649da3466beaf52'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:21.213570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f4b9821d-4fca-4eb0-97ac-896e9741eca8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c0cdba7439b40bbb649da3466beaf52'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:21.225975 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f4b9821d-4fca-4eb0-97ac-896e9741eca8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c0cdba7439b40bbb649da3466beaf52'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:21.226848 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f4b9821d-4fca-4eb0-97ac-896e9741eca8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c0cdba7439b40bbb649da3466beaf52'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:21.240242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 990/1982] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:20 2022] DELETE /identity/v3/users/2c0cdba7439b40bbb649da3466beaf52 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:21.590404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1797e-19eb-4cfb-9fa8-8282b72f945e service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:21.591810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8aa1797e-19eb-4cfb-9fa8-8282b72f945e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:21.611845 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa1797e-19eb-4cfb-9fa8-8282b72f945e service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:21.675875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449208d8-155b-4d6a-ac8d-23cfaf631a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:21.676501 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449208d8-155b-4d6a-ac8d-23cfaf631a47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:21.676805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449208d8-155b-4d6a-ac8d-23cfaf631a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:21.677074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449208d8-155b-4d6a-ac8d-23cfaf631a47 admin admin] PATH_INFO: `/v3/projects/fde17b6ad5944510950ce7bc2f034071` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:21.678316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449208d8-155b-4d6a-ac8d-23cfaf631a47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fde17b6ad5944510950ce7bc2f034071)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:21.680271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449208d8-155b-4d6a-ac8d-23cfaf631a47 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:21.731088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-449208d8-155b-4d6a-ac8d-23cfaf631a47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fde17b6ad5944510950ce7bc2f034071', 'request_id': 'req-449208d8-155b-4d6a-ac8d-23cfaf631a47'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:21.738753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-449208d8-155b-4d6a-ac8d-23cfaf631a47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fde17b6ad5944510950ce7bc2f034071', 'request_id': 'req-449208d8-155b-4d6a-ac8d-23cfaf631a47'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:21.751752 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-449208d8-155b-4d6a-ac8d-23cfaf631a47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fde17b6ad5944510950ce7bc2f034071', 'request_id': 'req-449208d8-155b-4d6a-ac8d-23cfaf631a47'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:21.755740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:21.766944 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-449208d8-155b-4d6a-ac8d-23cfaf631a47 admin admin] The token cache is being invalidate because project fde17b6ad5944510950ce7bc2f034071 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:21.767230 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-449208d8-155b-4d6a-ac8d-23cfaf631a47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:21.767997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-449208d8-155b-4d6a-ac8d-23cfaf631a47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:21.769567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 993/1983] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:21 2022] DELETE /identity/v3/projects/fde17b6ad5944510950ce7bc2f034071 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:21.775735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f4c57-9d82-442d-95c3-f56d41b1c2b2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:21.776376 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f4c57-9d82-442d-95c3-f56d41b1c2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:21.776376 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0f4c57-9d82-442d-95c3-f56d41b1c2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:21.797192 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9e0f4c57-9d82-442d-95c3-f56d41b1c2b2 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:21.798618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9e0f4c57-9d82-442d-95c3-f56d41b1c2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:21.867331 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 991/1984] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:21.873258 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325c1b7-aba5-4488-b845-6071695d986a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:21.874120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325c1b7-aba5-4488-b845-6071695d986a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:21.874427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6325c1b7-aba5-4488-b845-6071695d986a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:21.894294 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6325c1b7-aba5-4488-b845-6071695d986a None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:21.895512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6325c1b7-aba5-4488-b845-6071695d986a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:21.903410 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 994/1985] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:21.908648 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f4c57-9d82-442d-95c3-f56d41b1c2b2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:21.918590 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0f4c57-9d82-442d-95c3-f56d41b1c2b2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:21.923649 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8554a-aab2-4c21-81e0-d6181188e5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:21.924250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8554a-aab2-4c21-81e0-d6181188e5bb admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:21.924522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8554a-aab2-4c21-81e0-d6181188e5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:21.924789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba8554a-aab2-4c21-81e0-d6181188e5bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:21.925434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8554a-aab2-4c21-81e0-d6181188e5bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:21.925732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8554a-aab2-4c21-81e0-d6181188e5bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:21.927423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba8554a-aab2-4c21-81e0-d6181188e5bb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:21.939824 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 992/1986] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:21.944541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325c1b7-aba5-4488-b845-6071695d986a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:21.954527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6325c1b7-aba5-4488-b845-6071695d986a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:21.960715 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29753d7f-7440-4982-b8f4-817e766da615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:21.961961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29753d7f-7440-4982-b8f4-817e766da615 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:21.961961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29753d7f-7440-4982-b8f4-817e766da615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:21.961961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29753d7f-7440-4982-b8f4-817e766da615 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:21.962473 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29753d7f-7440-4982-b8f4-817e766da615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:21.964417 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29753d7f-7440-4982-b8f4-817e766da615 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:21.994644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 995/1987] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:21 2022] POST /identity/v3/projects => generated 381 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:21.999427 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8554a-aab2-4c21-81e0-d6181188e5bb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:22.009445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba8554a-aab2-4c21-81e0-d6181188e5bb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:22.065510 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221b0c9-5b1f-4011-9033-044fb4b7e747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:22.066141 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221b0c9-5b1f-4011-9033-044fb4b7e747 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:22.066434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221b0c9-5b1f-4011-9033-044fb4b7e747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:22.066880 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e221b0c9-5b1f-4011-9033-044fb4b7e747 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:22.068818 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221b0c9-5b1f-4011-9033-044fb4b7e747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:22.070645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e221b0c9-5b1f-4011-9033-044fb4b7e747 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:22.093527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 993/1988] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:21 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:22.097625 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29753d7f-7440-4982-b8f4-817e766da615 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:22.121214 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29753d7f-7440-4982-b8f4-817e766da615 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:22.133549 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5912e59-f784-4b0d-ae9b-40e00ce4027a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:22.134297 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5912e59-f784-4b0d-ae9b-40e00ce4027a admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:22.134675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5912e59-f784-4b0d-ae9b-40e00ce4027a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:22.135069 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5912e59-f784-4b0d-ae9b-40e00ce4027a admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:22.135831 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5912e59-f784-4b0d-ae9b-40e00ce4027a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:22.137499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5912e59-f784-4b0d-ae9b-40e00ce4027a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:22.150431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 996/1989] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:22.154313 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221b0c9-5b1f-4011-9033-044fb4b7e747 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:22.164847 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e221b0c9-5b1f-4011-9033-044fb4b7e747 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:22.170028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416d4889-4249-48b6-9907-4a8d5ddd2420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:22.170669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d4889-4249-48b6-9907-4a8d5ddd2420 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:22.170940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d4889-4249-48b6-9907-4a8d5ddd2420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:22.171207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416d4889-4249-48b6-9907-4a8d5ddd2420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:22.171839 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d4889-4249-48b6-9907-4a8d5ddd2420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:22.173571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416d4889-4249-48b6-9907-4a8d5ddd2420 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:22.186580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 994/1990] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:22.190757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5912e59-f784-4b0d-ae9b-40e00ce4027a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:22.200990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5912e59-f784-4b0d-ae9b-40e00ce4027a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:22.206129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57ff41-b984-4787-8f35-8c719d4e20b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:22.206821 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57ff41-b984-4787-8f35-8c719d4e20b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:22.207098 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57ff41-b984-4787-8f35-8c719d4e20b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:22.207367 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec57ff41-b984-4787-8f35-8c719d4e20b5 admin admin] PATH_INFO: `/v3/projects/3385b15fee124639a48f7c1baac22966/users/c917857028c44688afbbf1a6b60adc8f/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:22.225493 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57ff41-b984-4787-8f35-8c719d4e20b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3385b15fee124639a48f7c1baac22966, user_id=c917857028c44688afbbf1a6b60adc8f, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:22.228382 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec57ff41-b984-4787-8f35-8c719d4e20b5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:22.245557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 997/1991] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:22 2022] PUT /identity/v3/projects/3385b15fee124639a48f7c1baac22966/users/c917857028c44688afbbf1a6b60adc8f/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:22.256771 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1a014-b0f7-497e-baa4-d26a8a81da0c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:22.257152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1a014-b0f7-497e-baa4-d26a8a81da0c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:22.257489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1a014-b0f7-497e-baa4-d26a8a81da0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:22.274850 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:22.297539 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5bf1a014-b0f7-497e-baa4-d26a8a81da0c None None] MFA Rules not processed for user `c917857028c44688afbbf1a6b60adc8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:22.298904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5bf1a014-b0f7-497e-baa4-d26a8a81da0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:22.389904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 995/1992] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:22.401179 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57ff41-b984-4787-8f35-8c719d4e20b5 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:22.402831 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ec57ff41-b984-4787-8f35-8c719d4e20b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:22.465875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec57ff41-b984-4787-8f35-8c719d4e20b5 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:22.470053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937e0fa4-ea8f-45c7-9a6f-0452dbbdb103 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:22.470761 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e0fa4-ea8f-45c7-9a6f-0452dbbdb103 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:22.471105 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e0fa4-ea8f-45c7-9a6f-0452dbbdb103 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:22.471444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937e0fa4-ea8f-45c7-9a6f-0452dbbdb103 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:22.481611 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e0fa4-ea8f-45c7-9a6f-0452dbbdb103 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:22.483840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937e0fa4-ea8f-45c7-9a6f-0452dbbdb103 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:22.502271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 998/1993] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:22 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:32.386307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=29 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:32.492741 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=1,keystone:SELECT=55,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:34.381305 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1a014-b0f7-497e-baa4-d26a8a81da0c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:34.386096 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5bf1a014-b0f7-497e-baa4-d26a8a81da0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:34.470066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1a014-b0f7-497e-baa4-d26a8a81da0c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:34.476531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcfe04-a6b6-415f-9658-e27a5d0f5637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:34.477218 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcfe04-a6b6-415f-9658-e27a5d0f5637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:34.477658 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcfe04-a6b6-415f-9658-e27a5d0f5637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:34.477968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efcfe04-a6b6-415f-9658-e27a5d0f5637 admin admin] PATH_INFO: `/v3/users/e0f476a0b37c4d229777ac4207c74206` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:34.479163 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcfe04-a6b6-415f-9658-e27a5d0f5637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0f476a0b37c4d229777ac4207c74206)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:34.481121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efcfe04-a6b6-415f-9658-e27a5d0f5637 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:34.543565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0efcfe04-a6b6-415f-9658-e27a5d0f5637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0f476a0b37c4d229777ac4207c74206'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:34.544558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0efcfe04-a6b6-415f-9658-e27a5d0f5637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0f476a0b37c4d229777ac4207c74206'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:34.566872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0efcfe04-a6b6-415f-9658-e27a5d0f5637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0f476a0b37c4d229777ac4207c74206'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:34.579041 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0efcfe04-a6b6-415f-9658-e27a5d0f5637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0f476a0b37c4d229777ac4207c74206'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:34.580168 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0efcfe04-a6b6-415f-9658-e27a5d0f5637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0f476a0b37c4d229777ac4207c74206'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:34.599641 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 996/1994] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:34 2022] DELETE /identity/v3/users/e0f476a0b37c4d229777ac4207c74206 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:34.605237 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9775744-cf12-4f42-8f06-9a461b902d51 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:34.605834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9775744-cf12-4f42-8f06-9a461b902d51 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:34.605834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9775744-cf12-4f42-8f06-9a461b902d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:34.631167 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a9775744-cf12-4f42-8f06-9a461b902d51 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:34.635186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a9775744-cf12-4f42-8f06-9a461b902d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:34.796376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 999/1995] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:34.807457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcfe04-a6b6-415f-9658-e27a5d0f5637 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:34.809049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0efcfe04-a6b6-415f-9658-e27a5d0f5637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:34.904820 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efcfe04-a6b6-415f-9658-e27a5d0f5637 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:34.910193 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a9e72-d1ea-4415-8448-5f9ec4c9c80e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:34.911470 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a9e72-d1ea-4415-8448-5f9ec4c9c80e service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:34.912012 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a9e72-d1ea-4415-8448-5f9ec4c9c80e service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:34.912635 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a9e72-d1ea-4415-8448-5f9ec4c9c80e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:34.924656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a9e72-d1ea-4415-8448-5f9ec4c9c80e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:34.926149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3a9e72-d1ea-4415-8448-5f9ec4c9c80e service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:34.938999 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 997/1996] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:35.384754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9775744-cf12-4f42-8f06-9a461b902d51 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:35.386598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a9775744-cf12-4f42-8f06-9a461b902d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:35.401665 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a9e72-d1ea-4415-8448-5f9ec4c9c80e service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:35.405844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7b3a9e72-d1ea-4415-8448-5f9ec4c9c80e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:35.406002 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9775744-cf12-4f42-8f06-9a461b902d51 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:35.421520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204cc2a3-9e6b-41ba-a3e4-44881201d189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:35.422307 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204cc2a3-9e6b-41ba-a3e4-44881201d189 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:35.422659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204cc2a3-9e6b-41ba-a3e4-44881201d189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:35.422960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204cc2a3-9e6b-41ba-a3e4-44881201d189 admin admin] PATH_INFO: `/v3/users/be45cc3aa12e44a4b6d49976b6eff0b3` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:35.428220 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204cc2a3-9e6b-41ba-a3e4-44881201d189 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be45cc3aa12e44a4b6d49976b6eff0b3)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:35.430520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204cc2a3-9e6b-41ba-a3e4-44881201d189 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:35.460394 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a9e72-d1ea-4415-8448-5f9ec4c9c80e service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:35.480497 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726a7f8-ee03-4031-abbe-ea28a41866c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:35.481862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726a7f8-ee03-4031-abbe-ea28a41866c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:35.482412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726a7f8-ee03-4031-abbe-ea28a41866c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:35.483013 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726a7f8-ee03-4031-abbe-ea28a41866c6 admin admin] PATH_INFO: `/v3/users/c28048fb141449c3ae13c07069660a39` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:35.488754 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726a7f8-ee03-4031-abbe-ea28a41866c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c28048fb141449c3ae13c07069660a39)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:35.492525 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726a7f8-ee03-4031-abbe-ea28a41866c6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:35.558100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-204cc2a3-9e6b-41ba-a3e4-44881201d189 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be45cc3aa12e44a4b6d49976b6eff0b3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:35.558708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:35.582562 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-204cc2a3-9e6b-41ba-a3e4-44881201d189 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be45cc3aa12e44a4b6d49976b6eff0b3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:35.623185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8726a7f8-ee03-4031-abbe-ea28a41866c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c28048fb141449c3ae13c07069660a39'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:35.624117 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8726a7f8-ee03-4031-abbe-ea28a41866c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c28048fb141449c3ae13c07069660a39'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:35.632499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-204cc2a3-9e6b-41ba-a3e4-44881201d189 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be45cc3aa12e44a4b6d49976b6eff0b3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:35.632499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-204cc2a3-9e6b-41ba-a3e4-44881201d189 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be45cc3aa12e44a4b6d49976b6eff0b3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:35.643679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-204cc2a3-9e6b-41ba-a3e4-44881201d189 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be45cc3aa12e44a4b6d49976b6eff0b3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:35.651538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8726a7f8-ee03-4031-abbe-ea28a41866c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c28048fb141449c3ae13c07069660a39'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:35.656176 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1000/1997] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:35 2022] DELETE /identity/v3/users/be45cc3aa12e44a4b6d49976b6eff0b3 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:35.658992 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d38ec7-66df-4270-9a8b-bafcac3732fd None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:35.659288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d38ec7-66df-4270-9a8b-bafcac3732fd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:35.659541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d38ec7-66df-4270-9a8b-bafcac3732fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:35.664148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8726a7f8-ee03-4031-abbe-ea28a41866c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c28048fb141449c3ae13c07069660a39'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:35.664992 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8726a7f8-ee03-4031-abbe-ea28a41866c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c28048fb141449c3ae13c07069660a39'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:35.697537 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 998/1998] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:35 2022] DELETE /identity/v3/users/c28048fb141449c3ae13c07069660a39 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:35.704412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605273d-059c-4503-91b6-190e94c46e05 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:35.704412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605273d-059c-4503-91b6-190e94c46e05 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:35.704412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c605273d-059c-4503-91b6-190e94c46e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:35.721847 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f5d38ec7-66df-4270-9a8b-bafcac3732fd None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:35.723635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f5d38ec7-66df-4270-9a8b-bafcac3732fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:35.758160 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c605273d-059c-4503-91b6-190e94c46e05 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:35.760332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c605273d-059c-4503-91b6-190e94c46e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:35.849331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1001/1999] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:35.851423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d38ec7-66df-4270-9a8b-bafcac3732fd None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:35.854295 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f5d38ec7-66df-4270-9a8b-bafcac3732fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:35.885760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d38ec7-66df-4270-9a8b-bafcac3732fd None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:35.895038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859b5da-22be-413a-958b-4e8b0d7157fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:35.896068 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859b5da-22be-413a-958b-4e8b0d7157fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:35.896586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859b5da-22be-413a-958b-4e8b0d7157fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:35.897159 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c859b5da-22be-413a-958b-4e8b0d7157fe admin admin] PATH_INFO: `/v3/projects/1da088a132824ec79df8504fa1aa6a68` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:35.899278 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859b5da-22be-413a-958b-4e8b0d7157fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=1da088a132824ec79df8504fa1aa6a68)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:35.902102 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 999/2000] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:35.903129 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605273d-059c-4503-91b6-190e94c46e05 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:35.903337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c859b5da-22be-413a-958b-4e8b0d7157fe admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:35.906210 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c605273d-059c-4503-91b6-190e94c46e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:35.923638 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:36.040213 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c605273d-059c-4503-91b6-190e94c46e05 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:36.040509 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c859b5da-22be-413a-958b-4e8b0d7157fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1da088a132824ec79df8504fa1aa6a68', 'request_id': 'req-c859b5da-22be-413a-958b-4e8b0d7157fe'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.052855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c859b5da-22be-413a-958b-4e8b0d7157fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1da088a132824ec79df8504fa1aa6a68', 'request_id': 'req-c859b5da-22be-413a-958b-4e8b0d7157fe'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.065841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c859b5da-22be-413a-958b-4e8b0d7157fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1da088a132824ec79df8504fa1aa6a68', 'request_id': 'req-c859b5da-22be-413a-958b-4e8b0d7157fe'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.085587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c859b5da-22be-413a-958b-4e8b0d7157fe admin admin] The token cache is being invalidate because project 1da088a132824ec79df8504fa1aa6a68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:36.086038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c859b5da-22be-413a-958b-4e8b0d7157fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.086992 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c859b5da-22be-413a-958b-4e8b0d7157fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.088840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1002/2001] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:35 2022] DELETE /identity/v3/projects/1da088a132824ec79df8504fa1aa6a68 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:36.089429 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859b5da-22be-413a-958b-4e8b0d7157fe admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:36.090805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c859b5da-22be-413a-958b-4e8b0d7157fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:36.107422 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e4101-61cf-47ef-a389-bbcab81bc913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:36.109216 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e4101-61cf-47ef-a389-bbcab81bc913 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:36.109960 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e4101-61cf-47ef-a389-bbcab81bc913 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:36.109960 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74e4101-61cf-47ef-a389-bbcab81bc913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:36.111705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b74e4101-61cf-47ef-a389-bbcab81bc913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:36.117602 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:36.178932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c859b5da-22be-413a-958b-4e8b0d7157fe admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:36.182869 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d75b95-97ed-402f-b972-7e861799468a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:36.183489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d75b95-97ed-402f-b972-7e861799468a service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:36.183767 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d75b95-97ed-402f-b972-7e861799468a service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:36.184039 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d75b95-97ed-402f-b972-7e861799468a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:36.186531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-13d75b95-97ed-402f-b972-7e861799468a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:36.205751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e4101-61cf-47ef-a389-bbcab81bc913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:36.207243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74e4101-61cf-47ef-a389-bbcab81bc913 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:36.221100 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1000/2002] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:36.222703 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e4101-61cf-47ef-a389-bbcab81bc913 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:36.225036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b74e4101-61cf-47ef-a389-bbcab81bc913 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:36.266394 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74e4101-61cf-47ef-a389-bbcab81bc913 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:36.275769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe3b18-fab8-4df3-a337-9959a287c13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:36.276776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe3b18-fab8-4df3-a337-9959a287c13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:36.277352 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe3b18-fab8-4df3-a337-9959a287c13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:36.278622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe3b18-fab8-4df3-a337-9959a287c13f admin admin] PATH_INFO: `/v3/users/654cfaaed148401abf77df26ef22a85e` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:36.280967 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe3b18-fab8-4df3-a337-9959a287c13f admin admin] RBAC: Authorizing `identity:delete_user(user_id=654cfaaed148401abf77df26ef22a85e)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:36.284468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe3b18-fab8-4df3-a337-9959a287c13f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:36.307984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d75b95-97ed-402f-b972-7e861799468a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:36.309961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d75b95-97ed-402f-b972-7e861799468a service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:36.325330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1003/2003] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:36.355802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d75b95-97ed-402f-b972-7e861799468a service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:36.357573 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-13d75b95-97ed-402f-b972-7e861799468a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:36.378485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d75b95-97ed-402f-b972-7e861799468a service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:36.385989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d209591-5c3e-4b8b-8627-57b8fa27ea29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:36.386769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d209591-5c3e-4b8b-8627-57b8fa27ea29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:36.387085 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d209591-5c3e-4b8b-8627-57b8fa27ea29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:36.387370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d209591-5c3e-4b8b-8627-57b8fa27ea29 admin admin] PATH_INFO: `/v3/users/170dd6f119354f3dab51eb962bb032d7` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:36.389495 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d209591-5c3e-4b8b-8627-57b8fa27ea29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=170dd6f119354f3dab51eb962bb032d7)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:36.393657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d209591-5c3e-4b8b-8627-57b8fa27ea29 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:36.400345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dbfe3b18-fab8-4df3-a337-9959a287c13f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '654cfaaed148401abf77df26ef22a85e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.401477 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dbfe3b18-fab8-4df3-a337-9959a287c13f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '654cfaaed148401abf77df26ef22a85e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.420235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dbfe3b18-fab8-4df3-a337-9959a287c13f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '654cfaaed148401abf77df26ef22a85e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.430721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dbfe3b18-fab8-4df3-a337-9959a287c13f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '654cfaaed148401abf77df26ef22a85e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.431567 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-dbfe3b18-fab8-4df3-a337-9959a287c13f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '654cfaaed148401abf77df26ef22a85e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.457593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1001/2004] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:36 2022] DELETE /identity/v3/users/654cfaaed148401abf77df26ef22a85e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:36.468315 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:36.476080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5d209591-5c3e-4b8b-8627-57b8fa27ea29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '170dd6f119354f3dab51eb962bb032d7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.488483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5d209591-5c3e-4b8b-8627-57b8fa27ea29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '170dd6f119354f3dab51eb962bb032d7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.506393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5d209591-5c3e-4b8b-8627-57b8fa27ea29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '170dd6f119354f3dab51eb962bb032d7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.507327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5d209591-5c3e-4b8b-8627-57b8fa27ea29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '170dd6f119354f3dab51eb962bb032d7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.518676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5d209591-5c3e-4b8b-8627-57b8fa27ea29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '170dd6f119354f3dab51eb962bb032d7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.528761 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1004/2005] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:36 2022] DELETE /identity/v3/users/170dd6f119354f3dab51eb962bb032d7 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:36.534183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00179ca3-f59f-4c81-8429-4bd6d4235fa8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:36.534862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00179ca3-f59f-4c81-8429-4bd6d4235fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:36.535286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00179ca3-f59f-4c81-8429-4bd6d4235fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:36.559131 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-00179ca3-f59f-4c81-8429-4bd6d4235fa8 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:36.561279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-00179ca3-f59f-4c81-8429-4bd6d4235fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:36.652242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d209591-5c3e-4b8b-8627-57b8fa27ea29 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:36.659519 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5d209591-5c3e-4b8b-8627-57b8fa27ea29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:36.694150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1002/2006] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:36.718397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00179ca3-f59f-4c81-8429-4bd6d4235fa8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:36.720668 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-00179ca3-f59f-4c81-8429-4bd6d4235fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:36.739044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:36.775599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d209591-5c3e-4b8b-8627-57b8fa27ea29 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:36.785055 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162d301-caa0-49db-8849-74f886f06c05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:36.787100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162d301-caa0-49db-8849-74f886f06c05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:36.787430 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162d301-caa0-49db-8849-74f886f06c05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:36.787847 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3162d301-caa0-49db-8849-74f886f06c05 admin admin] PATH_INFO: `/v3/users/c917857028c44688afbbf1a6b60adc8f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:36.789881 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162d301-caa0-49db-8849-74f886f06c05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c917857028c44688afbbf1a6b60adc8f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:36.794183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3162d301-caa0-49db-8849-74f886f06c05 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:36.818586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00179ca3-f59f-4c81-8429-4bd6d4235fa8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:36.823704 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5342263e-4384-42b5-81ed-13763b7006fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:36.824544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342263e-4384-42b5-81ed-13763b7006fb service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:36.825160 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342263e-4384-42b5-81ed-13763b7006fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:36.825478 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5342263e-4384-42b5-81ed-13763b7006fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:36.842969 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342263e-4384-42b5-81ed-13763b7006fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:36.845701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5342263e-4384-42b5-81ed-13763b7006fb service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:36.860246 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1003/2007] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:36.870776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3162d301-caa0-49db-8849-74f886f06c05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c917857028c44688afbbf1a6b60adc8f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.887371 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3162d301-caa0-49db-8849-74f886f06c05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c917857028c44688afbbf1a6b60adc8f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.903739 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3162d301-caa0-49db-8849-74f886f06c05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c917857028c44688afbbf1a6b60adc8f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.905176 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3162d301-caa0-49db-8849-74f886f06c05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c917857028c44688afbbf1a6b60adc8f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.916036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3162d301-caa0-49db-8849-74f886f06c05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c917857028c44688afbbf1a6b60adc8f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:36.926357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1005/2008] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:36 2022] DELETE /identity/v3/users/c917857028c44688afbbf1a6b60adc8f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:36.930670 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf819ebe-4ade-46a2-a826-20ffdc18f3ce None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:36.931189 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf819ebe-4ade-46a2-a826-20ffdc18f3ce None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:36.931560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf819ebe-4ade-46a2-a826-20ffdc18f3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:36.964247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cf819ebe-4ade-46a2-a826-20ffdc18f3ce None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:36.965939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cf819ebe-4ade-46a2-a826-20ffdc18f3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:37.106529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1004/2009] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:37.118917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162d301-caa0-49db-8849-74f886f06c05 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:37.120695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3162d301-caa0-49db-8849-74f886f06c05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:37.131782 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf819ebe-4ade-46a2-a826-20ffdc18f3ce None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:37.136846 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cf819ebe-4ade-46a2-a826-20ffdc18f3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:37.168373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3162d301-caa0-49db-8849-74f886f06c05 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:37.182620 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2339ed-ab86-4f26-9f1e-885b47849215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:37.183271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2339ed-ab86-4f26-9f1e-885b47849215 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:37.183548 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2339ed-ab86-4f26-9f1e-885b47849215 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:37.183799 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2339ed-ab86-4f26-9f1e-885b47849215 admin admin] PATH_INFO: `/v3/projects/08285585867f4b89bf5548321022aa3e` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:37.186565 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2339ed-ab86-4f26-9f1e-885b47849215 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08285585867f4b89bf5548321022aa3e)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:37.188411 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2339ed-ab86-4f26-9f1e-885b47849215 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:37.283541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:37.302553 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fc2339ed-ab86-4f26-9f1e-885b47849215 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08285585867f4b89bf5548321022aa3e', 'request_id': 'req-fc2339ed-ab86-4f26-9f1e-885b47849215'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:37.312480 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fc2339ed-ab86-4f26-9f1e-885b47849215 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08285585867f4b89bf5548321022aa3e', 'request_id': 'req-fc2339ed-ab86-4f26-9f1e-885b47849215'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:37.324186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fc2339ed-ab86-4f26-9f1e-885b47849215 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08285585867f4b89bf5548321022aa3e', 'request_id': 'req-fc2339ed-ab86-4f26-9f1e-885b47849215'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:37.356858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fc2339ed-ab86-4f26-9f1e-885b47849215 admin admin] The token cache is being invalidate because project 08285585867f4b89bf5548321022aa3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:37.357184 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fc2339ed-ab86-4f26-9f1e-885b47849215 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:37.361472 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fc2339ed-ab86-4f26-9f1e-885b47849215 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:37.363867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1006/2010] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:37 2022] DELETE /identity/v3/projects/08285585867f4b89bf5548321022aa3e => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:37.366091 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf819ebe-4ade-46a2-a826-20ffdc18f3ce None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:37.367968 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2339ed-ab86-4f26-9f1e-885b47849215 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:37.369606 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fc2339ed-ab86-4f26-9f1e-885b47849215 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:37.462130 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f1315-5b25-4370-93d7-a979256966e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:37.462887 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f1315-5b25-4370-93d7-a979256966e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:37.463474 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f1315-5b25-4370-93d7-a979256966e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:37.463757 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f1315-5b25-4370-93d7-a979256966e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:37.471941 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4c4f1315-5b25-4370-93d7-a979256966e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:37.535953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2339ed-ab86-4f26-9f1e-885b47849215 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:37.542935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb40ca71-ed20-4a0a-9926-5308e087259e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:37.545297 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40ca71-ed20-4a0a-9926-5308e087259e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:37.546009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40ca71-ed20-4a0a-9926-5308e087259e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:37.546302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:37.546433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb40ca71-ed20-4a0a-9926-5308e087259e admin admin] PATH_INFO: `/v3/users/888d603152ea4ed29c5558b6763beb9c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:37.547900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb40ca71-ed20-4a0a-9926-5308e087259e admin admin] RBAC: Authorizing `identity:delete_user(user_id=888d603152ea4ed29c5558b6763beb9c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:37.552422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb40ca71-ed20-4a0a-9926-5308e087259e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:37.613413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f1315-5b25-4370-93d7-a979256966e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:37.619114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f1315-5b25-4370-93d7-a979256966e8 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:37.633543 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fb40ca71-ed20-4a0a-9926-5308e087259e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '888d603152ea4ed29c5558b6763beb9c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:37.645540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1005/2011] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:37.647410 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f1315-5b25-4370-93d7-a979256966e8 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:37.649108 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4c4f1315-5b25-4370-93d7-a979256966e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:37.652481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fb40ca71-ed20-4a0a-9926-5308e087259e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '888d603152ea4ed29c5558b6763beb9c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:37.687262 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fb40ca71-ed20-4a0a-9926-5308e087259e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '888d603152ea4ed29c5558b6763beb9c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:37.688288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fb40ca71-ed20-4a0a-9926-5308e087259e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '888d603152ea4ed29c5558b6763beb9c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:37.688971 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f1315-5b25-4370-93d7-a979256966e8 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:37.716019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5258cd-dd55-469a-8d73-60464e59def6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:37.720823 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5258cd-dd55-469a-8d73-60464e59def6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:37.722480 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5258cd-dd55-469a-8d73-60464e59def6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:37.723012 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5258cd-dd55-469a-8d73-60464e59def6 admin admin] PATH_INFO: `/v3/projects/1c2534f7d77541dc8798b47ae613863a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:37.723121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fb40ca71-ed20-4a0a-9926-5308e087259e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '888d603152ea4ed29c5558b6763beb9c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:37.725954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5258cd-dd55-469a-8d73-60464e59def6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c2534f7d77541dc8798b47ae613863a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:37.729439 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5258cd-dd55-469a-8d73-60464e59def6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:37.734815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1007/2012] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:37 2022] DELETE /identity/v3/users/888d603152ea4ed29c5558b6763beb9c => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:37.734815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb40ca71-ed20-4a0a-9926-5308e087259e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:37.734815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fb40ca71-ed20-4a0a-9926-5308e087259e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:37.822921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:37.874402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb40ca71-ed20-4a0a-9926-5308e087259e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:37.901849 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b5258cd-dd55-469a-8d73-60464e59def6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c2534f7d77541dc8798b47ae613863a', 'request_id': 'req-9b5258cd-dd55-469a-8d73-60464e59def6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:37.921147 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b5258cd-dd55-469a-8d73-60464e59def6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c2534f7d77541dc8798b47ae613863a', 'request_id': 'req-9b5258cd-dd55-469a-8d73-60464e59def6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:37.947957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992605c5-d3b9-493c-872f-6597ab53ada9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:37.948946 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992605c5-d3b9-493c-872f-6597ab53ada9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:37.949079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b5258cd-dd55-469a-8d73-60464e59def6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c2534f7d77541dc8798b47ae613863a', 'request_id': 'req-9b5258cd-dd55-469a-8d73-60464e59def6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:37.949562 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992605c5-d3b9-493c-872f-6597ab53ada9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:37.950028 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992605c5-d3b9-493c-872f-6597ab53ada9 admin admin] PATH_INFO: `/v3/projects/1abd098239644177ac3fda4d204ea6e1` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:37.951989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992605c5-d3b9-493c-872f-6597ab53ada9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1abd098239644177ac3fda4d204ea6e1)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:37.955388 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992605c5-d3b9-493c-872f-6597ab53ada9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:37.984094 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b5258cd-dd55-469a-8d73-60464e59def6 admin admin] The token cache is being invalidate because project 1c2534f7d77541dc8798b47ae613863a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:37.984635 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b5258cd-dd55-469a-8d73-60464e59def6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:37.987753 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9b5258cd-dd55-469a-8d73-60464e59def6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:37.991987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1006/2013] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:37 2022] DELETE /identity/v3/projects/1c2534f7d77541dc8798b47ae613863a => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:38.001443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a03e90-527e-4eb7-89f9-eb047ccbb3ba None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:38.001443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a03e90-527e-4eb7-89f9-eb047ccbb3ba None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:38.012606 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a03e90-527e-4eb7-89f9-eb047ccbb3ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:38.059790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-992605c5-d3b9-493c-872f-6597ab53ada9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1abd098239644177ac3fda4d204ea6e1', 'request_id': 'req-992605c5-d3b9-493c-872f-6597ab53ada9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.064535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-58a03e90-527e-4eb7-89f9-eb047ccbb3ba None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:38.065797 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-58a03e90-527e-4eb7-89f9-eb047ccbb3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:38.069841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-992605c5-d3b9-493c-872f-6597ab53ada9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1abd098239644177ac3fda4d204ea6e1', 'request_id': 'req-992605c5-d3b9-493c-872f-6597ab53ada9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.081725 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-992605c5-d3b9-493c-872f-6597ab53ada9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1abd098239644177ac3fda4d204ea6e1', 'request_id': 'req-992605c5-d3b9-493c-872f-6597ab53ada9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.118108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-992605c5-d3b9-493c-872f-6597ab53ada9 admin admin] The token cache is being invalidate because project 1abd098239644177ac3fda4d204ea6e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:38.118924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-992605c5-d3b9-493c-872f-6597ab53ada9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.123363 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-992605c5-d3b9-493c-872f-6597ab53ada9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.125852 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1008/2014] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:37 2022] DELETE /identity/v3/projects/1abd098239644177ac3fda4d204ea6e1 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:38.127011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992605c5-d3b9-493c-872f-6597ab53ada9 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:38.129058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-992605c5-d3b9-493c-872f-6597ab53ada9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:38.256168 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1007/2015] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:38.258157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a03e90-527e-4eb7-89f9-eb047ccbb3ba None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:38.261890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-58a03e90-527e-4eb7-89f9-eb047ccbb3ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:38.283996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a03e90-527e-4eb7-89f9-eb047ccbb3ba None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:38.289439 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54be15b7-c047-44cf-892b-1c9ec8da2277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:38.290527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be15b7-c047-44cf-892b-1c9ec8da2277 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:38.290910 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be15b7-c047-44cf-892b-1c9ec8da2277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:38.291691 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54be15b7-c047-44cf-892b-1c9ec8da2277 admin admin] PATH_INFO: `/v3/users/35077a1f2ac848deaa487bddba4eb763` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:38.292518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be15b7-c047-44cf-892b-1c9ec8da2277 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35077a1f2ac848deaa487bddba4eb763)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:38.295994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54be15b7-c047-44cf-892b-1c9ec8da2277 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:38.297226 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992605c5-d3b9-493c-872f-6597ab53ada9 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:38.325855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:38.326566 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:38.326853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:38.329481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf admin admin] PATH_INFO: `/v3/projects/ee689c95ffec4cdabd0c5fd83c82a753` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:38.337175 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee689c95ffec4cdabd0c5fd83c82a753)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:38.339409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:38.395379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:38.412377 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:38.426213 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-54be15b7-c047-44cf-892b-1c9ec8da2277 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35077a1f2ac848deaa487bddba4eb763'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.429329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-54be15b7-c047-44cf-892b-1c9ec8da2277 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35077a1f2ac848deaa487bddba4eb763'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.455137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-54be15b7-c047-44cf-892b-1c9ec8da2277 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35077a1f2ac848deaa487bddba4eb763'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.463481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee689c95ffec4cdabd0c5fd83c82a753', 'request_id': 'req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.470019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-54be15b7-c047-44cf-892b-1c9ec8da2277 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35077a1f2ac848deaa487bddba4eb763'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.471283 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-54be15b7-c047-44cf-892b-1c9ec8da2277 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35077a1f2ac848deaa487bddba4eb763'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.493768 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee689c95ffec4cdabd0c5fd83c82a753', 'request_id': 'req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.501659 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1008/2016] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:38 2022] DELETE /identity/v3/users/35077a1f2ac848deaa487bddba4eb763 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:38.506924 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f89b5-2934-46cc-bfce-95a109b5d05e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:38.507579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f89b5-2934-46cc-bfce-95a109b5d05e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:38.508019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9f89b5-2934-46cc-bfce-95a109b5d05e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:38.516547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee689c95ffec4cdabd0c5fd83c82a753', 'request_id': 'req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.556823 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf admin admin] The token cache is being invalidate because project ee689c95ffec4cdabd0c5fd83c82a753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:38.557130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.557859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.559579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1009/2017] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:38 2022] DELETE /identity/v3/projects/ee689c95ffec4cdabd0c5fd83c82a753 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:38.565226 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:38.565531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ac9f89b5-2934-46cc-bfce-95a109b5d05e None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:38.567979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:38.570546 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ac9f89b5-2934-46cc-bfce-95a109b5d05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:38.716490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1009/2018] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:38.717820 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f89b5-2934-46cc-bfce-95a109b5d05e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:38.719880 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ac9f89b5-2934-46cc-bfce-95a109b5d05e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:38.729418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e33c46c-f2e4-4de3-a299-59baec1ed4cf admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:38.737860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9f89b5-2934-46cc-bfce-95a109b5d05e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:38.750599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24990a50-b9cf-4f89-a588-11093fbdc91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:38.751275 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24990a50-b9cf-4f89-a588-11093fbdc91d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:38.751556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24990a50-b9cf-4f89-a588-11093fbdc91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:38.751892 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24990a50-b9cf-4f89-a588-11093fbdc91d admin admin] PATH_INFO: `/v3/users/3c165cf8aba64721a9bcd47a80809d63` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:38.753429 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24990a50-b9cf-4f89-a588-11093fbdc91d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c165cf8aba64721a9bcd47a80809d63)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:38.754631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46d1bf-b092-462e-b68c-c8c94e295500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:38.755659 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46d1bf-b092-462e-b68c-c8c94e295500 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:38.755852 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24990a50-b9cf-4f89-a588-11093fbdc91d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:38.756163 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46d1bf-b092-462e-b68c-c8c94e295500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:38.756843 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46d1bf-b092-462e-b68c-c8c94e295500 admin admin] PATH_INFO: `/v3/projects/d62f5051ce9d4486a0fdb807a9810da1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:38.772764 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46d1bf-b092-462e-b68c-c8c94e295500 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d62f5051ce9d4486a0fdb807a9810da1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:38.787657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46d1bf-b092-462e-b68c-c8c94e295500 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:38.859838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-24990a50-b9cf-4f89-a588-11093fbdc91d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c165cf8aba64721a9bcd47a80809d63'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.880927 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-24990a50-b9cf-4f89-a588-11093fbdc91d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c165cf8aba64721a9bcd47a80809d63'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.914154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-24990a50-b9cf-4f89-a588-11093fbdc91d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c165cf8aba64721a9bcd47a80809d63'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.916119 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-24990a50-b9cf-4f89-a588-11093fbdc91d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c165cf8aba64721a9bcd47a80809d63'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.924951 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2d46d1bf-b092-462e-b68c-c8c94e295500 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd62f5051ce9d4486a0fdb807a9810da1', 'request_id': 'req-2d46d1bf-b092-462e-b68c-c8c94e295500'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.927485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-24990a50-b9cf-4f89-a588-11093fbdc91d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c165cf8aba64721a9bcd47a80809d63'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.932977 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2d46d1bf-b092-462e-b68c-c8c94e295500 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd62f5051ce9d4486a0fdb807a9810da1', 'request_id': 'req-2d46d1bf-b092-462e-b68c-c8c94e295500'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.940667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1010/2019] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:38 2022] DELETE /identity/v3/users/3c165cf8aba64721a9bcd47a80809d63 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:38.941679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24990a50-b9cf-4f89-a588-11093fbdc91d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:38.943081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-24990a50-b9cf-4f89-a588-11093fbdc91d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:38.944949 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2d46d1bf-b092-462e-b68c-c8c94e295500 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd62f5051ce9d4486a0fdb807a9810da1', 'request_id': 'req-2d46d1bf-b092-462e-b68c-c8c94e295500'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.969473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2d46d1bf-b092-462e-b68c-c8c94e295500 admin admin] The token cache is being invalidate because project d62f5051ce9d4486a0fdb807a9810da1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:38.969947 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2d46d1bf-b092-462e-b68c-c8c94e295500 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.971121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2d46d1bf-b092-462e-b68c-c8c94e295500 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:38.974512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1010/2020] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:38 2022] DELETE /identity/v3/projects/d62f5051ce9d4486a0fdb807a9810da1 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:38.975393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46d1bf-b092-462e-b68c-c8c94e295500 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:38.977044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2d46d1bf-b092-462e-b68c-c8c94e295500 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:38.985973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:39.048358 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24990a50-b9cf-4f89-a588-11093fbdc91d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:39.061818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a134ac-baa2-4090-b896-6c0bb464f5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:39.062553 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a134ac-baa2-4090-b896-6c0bb464f5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:39.062962 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a134ac-baa2-4090-b896-6c0bb464f5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:39.063233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a134ac-baa2-4090-b896-6c0bb464f5a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:39.063907 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a134ac-baa2-4090-b896-6c0bb464f5a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:39.064712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a134ac-baa2-4090-b896-6c0bb464f5a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:39.065814 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a134ac-baa2-4090-b896-6c0bb464f5a7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:39.088022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1011/2021] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:39.089546 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:39.093033 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43f3e-ea4b-44ff-a8b1-c3991eb528b1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:39.093632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43f3e-ea4b-44ff-a8b1-c3991eb528b1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:39.094481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e43f3e-ea4b-44ff-a8b1-c3991eb528b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:39.128222 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46d1bf-b092-462e-b68c-c8c94e295500 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:39.135722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a0e43f3e-ea4b-44ff-a8b1-c3991eb528b1 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:39.137917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a0e43f3e-ea4b-44ff-a8b1-c3991eb528b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:39.138611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:39.139643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:39.140300 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:39.140762 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2 admin admin] PATH_INFO: `/v3/projects/3385b15fee124639a48f7c1baac22966` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:39.143192 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3385b15fee124639a48f7c1baac22966)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:39.146097 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:39.154875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1012/2022] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:39.156494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e43f3e-ea4b-44ff-a8b1-c3991eb528b1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:39.159130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a0e43f3e-ea4b-44ff-a8b1-c3991eb528b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:39.180915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e43f3e-ea4b-44ff-a8b1-c3991eb528b1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:39.191080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e001a5-bd44-4024-8de5-3646564f1837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:39.192160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e001a5-bd44-4024-8de5-3646564f1837 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:39.192660 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e001a5-bd44-4024-8de5-3646564f1837 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:39.193446 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e001a5-bd44-4024-8de5-3646564f1837 admin admin] PATH_INFO: `/v3/projects/dbdec56ef7f04398abaca0511e5ed300` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:39.195524 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e001a5-bd44-4024-8de5-3646564f1837 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbdec56ef7f04398abaca0511e5ed300)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:39.199254 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e001a5-bd44-4024-8de5-3646564f1837 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:39.209306 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3385b15fee124639a48f7c1baac22966', 'request_id': 'req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.222141 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3385b15fee124639a48f7c1baac22966', 'request_id': 'req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.234502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3385b15fee124639a48f7c1baac22966', 'request_id': 'req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.274838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2 admin admin] The token cache is being invalidate because project 3385b15fee124639a48f7c1baac22966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:39.275470 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.276960 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.280564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1011/2023] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:38 2022] DELETE /identity/v3/projects/3385b15fee124639a48f7c1baac22966 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:39.281937 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:39.286618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:39.296261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-47e001a5-bd44-4024-8de5-3646564f1837 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbdec56ef7f04398abaca0511e5ed300', 'request_id': 'req-47e001a5-bd44-4024-8de5-3646564f1837'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.306604 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-47e001a5-bd44-4024-8de5-3646564f1837 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbdec56ef7f04398abaca0511e5ed300', 'request_id': 'req-47e001a5-bd44-4024-8de5-3646564f1837'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.322032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-47e001a5-bd44-4024-8de5-3646564f1837 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbdec56ef7f04398abaca0511e5ed300', 'request_id': 'req-47e001a5-bd44-4024-8de5-3646564f1837'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.340736 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-47e001a5-bd44-4024-8de5-3646564f1837 admin admin] The token cache is being invalidate because project dbdec56ef7f04398abaca0511e5ed300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:39.341169 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-47e001a5-bd44-4024-8de5-3646564f1837 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.342725 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-47e001a5-bd44-4024-8de5-3646564f1837 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.344591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1013/2024] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:39 2022] DELETE /identity/v3/projects/dbdec56ef7f04398abaca0511e5ed300 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:39.345973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e04647-eabd-4dd6-95d5-9fdc87ecd500 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:39.346299 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e04647-eabd-4dd6-95d5-9fdc87ecd500 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:39.346572 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e04647-eabd-4dd6-95d5-9fdc87ecd500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:39.374788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a6e04647-eabd-4dd6-95d5-9fdc87ecd500 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:39.376035 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a6e04647-eabd-4dd6-95d5-9fdc87ecd500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:39.398387 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f56d96-7671-4ba2-957d-bc33a3f76ff2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:39.504477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1014/2025] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:39.507916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c60fa-a220-45fb-a53c-f98f98cd9280 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:39.508088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c60fa-a220-45fb-a53c-f98f98cd9280 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:39.508360 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c60fa-a220-45fb-a53c-f98f98cd9280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:39.516150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da88a0c-781a-44d3-9ca0-41b0408cf740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:39.517154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da88a0c-781a-44d3-9ca0-41b0408cf740 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:39.517662 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da88a0c-781a-44d3-9ca0-41b0408cf740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:39.518160 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da88a0c-781a-44d3-9ca0-41b0408cf740 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:39.519273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da88a0c-781a-44d3-9ca0-41b0408cf740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:39.523379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da88a0c-781a-44d3-9ca0-41b0408cf740 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:39.534601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-311c60fa-a220-45fb-a53c-f98f98cd9280 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:39.536091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-311c60fa-a220-45fb-a53c-f98f98cd9280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:39.553400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1015/2026] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:39.553400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c60fa-a220-45fb-a53c-f98f98cd9280 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:39.554736 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-311c60fa-a220-45fb-a53c-f98f98cd9280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:39.570842 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c60fa-a220-45fb-a53c-f98f98cd9280 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:39.572688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1012/2027] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:39 2022] POST /identity/v3/projects => generated 367 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:39.574187 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da88a0c-781a-44d3-9ca0-41b0408cf740 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:39.577115 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9da88a0c-781a-44d3-9ca0-41b0408cf740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:39.585195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:39.609450 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:39.653413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7a9ac-f22d-4fbd-b4cf-9b80d44d6704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:39.654591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7a9ac-f22d-4fbd-b4cf-9b80d44d6704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:39.654983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7a9ac-f22d-4fbd-b4cf-9b80d44d6704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:39.655328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd7a9ac-f22d-4fbd-b4cf-9b80d44d6704 admin admin] PATH_INFO: `/v3/users/2ec90633aa1341f6932ca355f2ecfb84` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:39.656769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7a9ac-f22d-4fbd-b4cf-9b80d44d6704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ec90633aa1341f6932ca355f2ecfb84)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:39.659133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd7a9ac-f22d-4fbd-b4cf-9b80d44d6704 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:39.674313 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da88a0c-781a-44d3-9ca0-41b0408cf740 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:39.679962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b644376-56cb-4354-ba5c-10f4c3f0062b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:39.680612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b644376-56cb-4354-ba5c-10f4c3f0062b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:39.680957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b644376-56cb-4354-ba5c-10f4c3f0062b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:39.681211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b644376-56cb-4354-ba5c-10f4c3f0062b admin admin] PATH_INFO: `/v3/projects/8e51610cbd714848a333d8f1b1ef73cf` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:39.682393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b644376-56cb-4354-ba5c-10f4c3f0062b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e51610cbd714848a333d8f1b1ef73cf)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:39.684334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b644376-56cb-4354-ba5c-10f4c3f0062b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:39.715470 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cbd7a9ac-f22d-4fbd-b4cf-9b80d44d6704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ec90633aa1341f6932ca355f2ecfb84'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.722631 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cbd7a9ac-f22d-4fbd-b4cf-9b80d44d6704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ec90633aa1341f6932ca355f2ecfb84'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.736935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cbd7a9ac-f22d-4fbd-b4cf-9b80d44d6704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ec90633aa1341f6932ca355f2ecfb84'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.738291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cbd7a9ac-f22d-4fbd-b4cf-9b80d44d6704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ec90633aa1341f6932ca355f2ecfb84'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.740425 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5b644376-56cb-4354-ba5c-10f4c3f0062b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e51610cbd714848a333d8f1b1ef73cf', 'request_id': 'req-5b644376-56cb-4354-ba5c-10f4c3f0062b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.751098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5b644376-56cb-4354-ba5c-10f4c3f0062b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e51610cbd714848a333d8f1b1ef73cf', 'request_id': 'req-5b644376-56cb-4354-ba5c-10f4c3f0062b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.752296 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cbd7a9ac-f22d-4fbd-b4cf-9b80d44d6704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ec90633aa1341f6932ca355f2ecfb84'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.761067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5b644376-56cb-4354-ba5c-10f4c3f0062b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e51610cbd714848a333d8f1b1ef73cf', 'request_id': 'req-5b644376-56cb-4354-ba5c-10f4c3f0062b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.764759 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1016/2028] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:39 2022] DELETE /identity/v3/users/2ec90633aa1341f6932ca355f2ecfb84 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:39.765914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7a9ac-f22d-4fbd-b4cf-9b80d44d6704 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:39.771086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cbd7a9ac-f22d-4fbd-b4cf-9b80d44d6704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:39.777797 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5b644376-56cb-4354-ba5c-10f4c3f0062b admin admin] The token cache is being invalidate because project 8e51610cbd714848a333d8f1b1ef73cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:39.778096 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5b644376-56cb-4354-ba5c-10f4c3f0062b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.778848 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5b644376-56cb-4354-ba5c-10f4c3f0062b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:39.783493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1013/2029] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:39 2022] DELETE /identity/v3/projects/8e51610cbd714848a333d8f1b1ef73cf => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:39.783493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dfc42-30ab-4ca6-aaa2-5ebcb668cb38 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:39.783493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dfc42-30ab-4ca6-aaa2-5ebcb668cb38 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:39.783493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287dfc42-30ab-4ca6-aaa2-5ebcb668cb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:39.807079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-287dfc42-30ab-4ca6-aaa2-5ebcb668cb38 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:39.808444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-287dfc42-30ab-4ca6-aaa2-5ebcb668cb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:39.860068 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd7a9ac-f22d-4fbd-b4cf-9b80d44d6704 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:39.866305 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c177e0a-5e55-4ee9-9c65-c65831c0cb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:39.866952 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c177e0a-5e55-4ee9-9c65-c65831c0cb65 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:39.867822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c177e0a-5e55-4ee9-9c65-c65831c0cb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:39.868113 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c177e0a-5e55-4ee9-9c65-c65831c0cb65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:39.868805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c177e0a-5e55-4ee9-9c65-c65831c0cb65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:39.869105 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c177e0a-5e55-4ee9-9c65-c65831c0cb65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:39.870892 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c177e0a-5e55-4ee9-9c65-c65831c0cb65 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:39.885170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1017/2030] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:39.889441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1014/2031] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:39.889441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dfc42-30ab-4ca6-aaa2-5ebcb668cb38 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:39.890669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c177e0a-5e55-4ee9-9c65-c65831c0cb65 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:39.892364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8c177e0a-5e55-4ee9-9c65-c65831c0cb65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:39.902137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-287dfc42-30ab-4ca6-aaa2-5ebcb668cb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:39.908754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c177e0a-5e55-4ee9-9c65-c65831c0cb65 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:39.913935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698c26b-0927-4dab-b406-427a008378fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:39.914903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698c26b-0927-4dab-b406-427a008378fa admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:39.915187 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698c26b-0927-4dab-b406-427a008378fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:39.915459 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0698c26b-0927-4dab-b406-427a008378fa admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:39.916088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698c26b-0927-4dab-b406-427a008378fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:39.918085 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0698c26b-0927-4dab-b406-427a008378fa admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:39.920850 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287dfc42-30ab-4ca6-aaa2-5ebcb668cb38 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:39.928704 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b927699-f596-4d43-81ed-23a6df6d915e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:39.929525 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b927699-f596-4d43-81ed-23a6df6d915e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:39.929903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b927699-f596-4d43-81ed-23a6df6d915e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:39.930336 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b927699-f596-4d43-81ed-23a6df6d915e admin admin] PATH_INFO: `/v3/users/81bebea949bf4df3acf117abd9e45349` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:39.931694 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b927699-f596-4d43-81ed-23a6df6d915e admin admin] RBAC: Authorizing `identity:delete_user(user_id=81bebea949bf4df3acf117abd9e45349)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:39.934709 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b927699-f596-4d43-81ed-23a6df6d915e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:39.955925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1018/2032] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:39 2022] POST /identity/v3/users => generated 284 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:39.958290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698c26b-0927-4dab-b406-427a008378fa admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:39.962433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0698c26b-0927-4dab-b406-427a008378fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:39.976280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0698c26b-0927-4dab-b406-427a008378fa admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:39.983333 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ec01e-61f1-48dd-bf93-179761c4260e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:39.984318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ec01e-61f1-48dd-bf93-179761c4260e admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:39.984442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ec01e-61f1-48dd-bf93-179761c4260e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:39.984663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ec01e-61f1-48dd-bf93-179761c4260e admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:39.985544 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ec01e-61f1-48dd-bf93-179761c4260e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:39.987659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ec01e-61f1-48dd-bf93-179761c4260e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:39.998585 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:40.021066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5b927699-f596-4d43-81ed-23a6df6d915e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81bebea949bf4df3acf117abd9e45349'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:40.021964 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5b927699-f596-4d43-81ed-23a6df6d915e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81bebea949bf4df3acf117abd9e45349'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:40.032377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1019/2033] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:39 2022] POST /identity/v3/projects => generated 379 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:40.033497 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ec01e-61f1-48dd-bf93-179761c4260e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:40.041142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5b927699-f596-4d43-81ed-23a6df6d915e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81bebea949bf4df3acf117abd9e45349'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:40.042035 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:40.053423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5b927699-f596-4d43-81ed-23a6df6d915e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81bebea949bf4df3acf117abd9e45349'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:40.054460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5b927699-f596-4d43-81ed-23a6df6d915e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81bebea949bf4df3acf117abd9e45349'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:40.073551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1015/2034] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:39 2022] DELETE /identity/v3/users/81bebea949bf4df3acf117abd9e45349 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:40.075195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b927699-f596-4d43-81ed-23a6df6d915e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:40.079431 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5b927699-f596-4d43-81ed-23a6df6d915e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:40.142872 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ec01e-61f1-48dd-bf93-179761c4260e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:40.183514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b927699-f596-4d43-81ed-23a6df6d915e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:40.190107 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3cda7-b84e-4bd1-ae0d-4c3f88a894f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:40.191022 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3cda7-b84e-4bd1-ae0d-4c3f88a894f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.191308 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3cda7-b84e-4bd1-ae0d-4c3f88a894f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.191921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b3cda7-b84e-4bd1-ae0d-4c3f88a894f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.192885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3cda7-b84e-4bd1-ae0d-4c3f88a894f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:40.195212 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b3cda7-b84e-4bd1-ae0d-4c3f88a894f7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:40.218589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1016/2035] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:40.219867 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3cda7-b84e-4bd1-ae0d-4c3f88a894f7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:40.223012 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-63b3cda7-b84e-4bd1-ae0d-4c3f88a894f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:40.238698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b3cda7-b84e-4bd1-ae0d-4c3f88a894f7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:40.244473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f415517-3f2b-42b2-bed4-66a32416288c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:40.245673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f415517-3f2b-42b2-bed4-66a32416288c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.246105 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f415517-3f2b-42b2-bed4-66a32416288c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.246475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f415517-3f2b-42b2-bed4-66a32416288c admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.247462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f415517-3f2b-42b2-bed4-66a32416288c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:40.249827 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f415517-3f2b-42b2-bed4-66a32416288c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:40.280703 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1017/2036] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:40 2022] POST /identity/v3/users => generated 290 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:40.283403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e6309-3b6f-48ff-b269-ad26919f28e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:40.284150 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e6309-3b6f-48ff-b269-ad26919f28e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.284601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e6309-3b6f-48ff-b269-ad26919f28e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.284882 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e6309-3b6f-48ff-b269-ad26919f28e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.284996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f415517-3f2b-42b2-bed4-66a32416288c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:40.286615 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e6309-3b6f-48ff-b269-ad26919f28e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:40.286870 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9f415517-3f2b-42b2-bed4-66a32416288c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:40.286977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e6309-3b6f-48ff-b269-ad26919f28e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:40.288875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e6309-3b6f-48ff-b269-ad26919f28e8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:40.304709 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f415517-3f2b-42b2-bed4-66a32416288c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:40.314138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1020/2037] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:40.314138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e6309-3b6f-48ff-b269-ad26919f28e8 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:40.320980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:40.321599 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.321884 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.322156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120 admin admin] PATH_INFO: `/v3/projects/d6321ab627154e5d894f64c1aae7daa2` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.324915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6321ab627154e5d894f64c1aae7daa2)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:40.327754 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:40.335406 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e6309-3b6f-48ff-b269-ad26919f28e8 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:40.349590 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e093e7-66ac-4a40-b214-a91a4373dd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:40.350238 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e093e7-66ac-4a40-b214-a91a4373dd49 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.351393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e093e7-66ac-4a40-b214-a91a4373dd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.351754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e093e7-66ac-4a40-b214-a91a4373dd49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.352457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e093e7-66ac-4a40-b214-a91a4373dd49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:40.356785 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e093e7-66ac-4a40-b214-a91a4373dd49 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:40.378959 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1021/2038] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:40.380025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e093e7-66ac-4a40-b214-a91a4373dd49 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:40.392748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e093e7-66ac-4a40-b214-a91a4373dd49 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:40.408444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77465c3e-b717-4ab1-8758-6b894e7992d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:40.408444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77465c3e-b717-4ab1-8758-6b894e7992d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.408444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77465c3e-b717-4ab1-8758-6b894e7992d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.408444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77465c3e-b717-4ab1-8758-6b894e7992d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.408924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77465c3e-b717-4ab1-8758-6b894e7992d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:40.408924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77465c3e-b717-4ab1-8758-6b894e7992d6 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:40.432641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1022/2039] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:40.434153 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6321ab627154e5d894f64c1aae7daa2', 'request_id': 'req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:40.437357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77465c3e-b717-4ab1-8758-6b894e7992d6 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:40.440064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-77465c3e-b717-4ab1-8758-6b894e7992d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:40.445008 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6321ab627154e5d894f64c1aae7daa2', 'request_id': 'req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:40.460279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6321ab627154e5d894f64c1aae7daa2', 'request_id': 'req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:40.460529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77465c3e-b717-4ab1-8758-6b894e7992d6 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:40.480437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120 admin admin] The token cache is being invalidate because project d6321ab627154e5d894f64c1aae7daa2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:40.480729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:40.481577 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:40.490409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1018/2040] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:40 2022] DELETE /identity/v3/projects/d6321ab627154e5d894f64c1aae7daa2 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:40.490409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:40.490409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:40.500457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=83,keystone:DELETE=12,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:40.578621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba5d65a-4e85-4bd4-bab1-8d756f85a120 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:40.579191 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fe306-11ef-4b7d-8b87-0d2a3293841e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:40.579704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fe306-11ef-4b7d-8b87-0d2a3293841e admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.579987 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fe306-11ef-4b7d-8b87-0d2a3293841e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.580259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372fe306-11ef-4b7d-8b87-0d2a3293841e admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.581617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372fe306-11ef-4b7d-8b87-0d2a3293841e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:40.583187 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372fe306-11ef-4b7d-8b87-0d2a3293841e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:40.585139 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496f847-55f4-43d0-b6ee-3a7c6239489d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:40.585939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f847-55f4-43d0-b6ee-3a7c6239489d admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.586336 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f847-55f4-43d0-b6ee-3a7c6239489d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.586659 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496f847-55f4-43d0-b6ee-3a7c6239489d admin admin] PATH_INFO: `/v3/projects/a8401139077d4e009a5597f0ba00e539/users/f8316cc52e4e497f9df463395cd12dd1/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.612372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496f847-55f4-43d0-b6ee-3a7c6239489d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8401139077d4e009a5597f0ba00e539, user_id=f8316cc52e4e497f9df463395cd12dd1, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:40.615312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496f847-55f4-43d0-b6ee-3a7c6239489d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:40.620483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1023/2041] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:40 2022] POST /identity/v3/projects => generated 375 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:40.621344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fe306-11ef-4b7d-8b87-0d2a3293841e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:40.624686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-372fe306-11ef-4b7d-8b87-0d2a3293841e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:40.637411 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:40.638962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1019/2042] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:40 2022] PUT /identity/v3/projects/a8401139077d4e009a5597f0ba00e539/users/f8316cc52e4e497f9df463395cd12dd1/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:40.640234 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99de6f1-6642-41ad-a5f8-1a9a747a347c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.640511 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99de6f1-6642-41ad-a5f8-1a9a747a347c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.640777 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99de6f1-6642-41ad-a5f8-1a9a747a347c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.668555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a99de6f1-6642-41ad-a5f8-1a9a747a347c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:40.670575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a99de6f1-6642-41ad-a5f8-1a9a747a347c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:40.712186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372fe306-11ef-4b7d-8b87-0d2a3293841e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:40.738886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1020/2043] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:40.740367 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99de6f1-6642-41ad-a5f8-1a9a747a347c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:40.741717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a99de6f1-6642-41ad-a5f8-1a9a747a347c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:40.756402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99de6f1-6642-41ad-a5f8-1a9a747a347c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:40.763287 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f8757b-a75b-45a3-a122-921b7cce9559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:40.764286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8757b-a75b-45a3-a122-921b7cce9559 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.764726 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8757b-a75b-45a3-a122-921b7cce9559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.765218 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f8757b-a75b-45a3-a122-921b7cce9559 admin admin] PATH_INFO: `/v3/projects/e18f121e4ac74e3a916aa09b3174e53c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.766952 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8757b-a75b-45a3-a122-921b7cce9559 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e18f121e4ac74e3a916aa09b3174e53c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:40.769007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f8757b-a75b-45a3-a122-921b7cce9559 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:40.778521 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d34cec-0b3f-45fe-a455-d56942074a0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:40.779174 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d34cec-0b3f-45fe-a455-d56942074a0d admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.779450 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d34cec-0b3f-45fe-a455-d56942074a0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.779717 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d34cec-0b3f-45fe-a455-d56942074a0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.780360 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d34cec-0b3f-45fe-a455-d56942074a0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:40.781949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d34cec-0b3f-45fe-a455-d56942074a0d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:40.792595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1024/2044] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:40.795350 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d34cec-0b3f-45fe-a455-d56942074a0d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:40.798333 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a6d34cec-0b3f-45fe-a455-d56942074a0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:40.812195 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d34cec-0b3f-45fe-a455-d56942074a0d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:40.830175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-72f8757b-a75b-45a3-a122-921b7cce9559 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e18f121e4ac74e3a916aa09b3174e53c', 'request_id': 'req-72f8757b-a75b-45a3-a122-921b7cce9559'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:40.835927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-72f8757b-a75b-45a3-a122-921b7cce9559 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e18f121e4ac74e3a916aa09b3174e53c', 'request_id': 'req-72f8757b-a75b-45a3-a122-921b7cce9559'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:40.843687 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-72f8757b-a75b-45a3-a122-921b7cce9559 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e18f121e4ac74e3a916aa09b3174e53c', 'request_id': 'req-72f8757b-a75b-45a3-a122-921b7cce9559'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:40.855950 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-72f8757b-a75b-45a3-a122-921b7cce9559 admin admin] The token cache is being invalidate because project e18f121e4ac74e3a916aa09b3174e53c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:40.856245 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-72f8757b-a75b-45a3-a122-921b7cce9559 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:40.856900 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-72f8757b-a75b-45a3-a122-921b7cce9559 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:40.859049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1021/2045] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:40 2022] DELETE /identity/v3/projects/e18f121e4ac74e3a916aa09b3174e53c => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:40.860244 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298acc8a-d1b2-462d-89e7-5434ddc2301c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.860531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298acc8a-d1b2-462d-89e7-5434ddc2301c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.860788 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298acc8a-d1b2-462d-89e7-5434ddc2301c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.881609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f261d6-9f65-481d-8fa7-ff1a3c76b163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:40.882250 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f261d6-9f65-481d-8fa7-ff1a3c76b163 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.882540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f261d6-9f65-481d-8fa7-ff1a3c76b163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.882946 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f261d6-9f65-481d-8fa7-ff1a3c76b163 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.883687 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f261d6-9f65-481d-8fa7-ff1a3c76b163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:40.885784 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f261d6-9f65-481d-8fa7-ff1a3c76b163 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:40.895539 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-298acc8a-d1b2-462d-89e7-5434ddc2301c None None] MFA Rules not processed for user `f8316cc52e4e497f9df463395cd12dd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:40.896754 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-298acc8a-d1b2-462d-89e7-5434ddc2301c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:40.901346 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:40.909660 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1025/2046] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:40 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:40.911242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556d50b-525a-455d-b6c7-ae65b2654319 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.911437 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556d50b-525a-455d-b6c7-ae65b2654319 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.911694 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4556d50b-525a-455d-b6c7-ae65b2654319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.932668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4556d50b-525a-455d-b6c7-ae65b2654319 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:40.933847 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4556d50b-525a-455d-b6c7-ae65b2654319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:40.945536 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1026/2047] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:40.946559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556d50b-525a-455d-b6c7-ae65b2654319 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:40.948391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4556d50b-525a-455d-b6c7-ae65b2654319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:40.961207 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4556d50b-525a-455d-b6c7-ae65b2654319 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:40.967164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09be91-980d-4c7e-b2ba-c4522345b729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:40.967835 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09be91-980d-4c7e-b2ba-c4522345b729 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.968116 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09be91-980d-4c7e-b2ba-c4522345b729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.968392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da09be91-980d-4c7e-b2ba-c4522345b729 admin admin] PATH_INFO: `/v3/projects/643f162d549042a6b9b1dc635b1c85e4/users/869317d364c2444693402272266ec73a/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.974325 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1022/2048] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:40 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:40.975584 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f815733-cd83-4339-8228-37c80974d55f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:40.975868 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f815733-cd83-4339-8228-37c80974d55f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:40.976141 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f815733-cd83-4339-8228-37c80974d55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:40.986204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09be91-980d-4c7e-b2ba-c4522345b729 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643f162d549042a6b9b1dc635b1c85e4, user_id=869317d364c2444693402272266ec73a, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:40.988945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da09be91-980d-4c7e-b2ba-c4522345b729 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:40.996587 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2f815733-cd83-4339-8228-37c80974d55f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:40.998597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2f815733-cd83-4339-8228-37c80974d55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:41.003737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1027/2049] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:40 2022] PUT /identity/v3/projects/643f162d549042a6b9b1dc635b1c85e4/users/869317d364c2444693402272266ec73a/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:41.004678 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09be91-980d-4c7e-b2ba-c4522345b729 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.006450 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-da09be91-980d-4c7e-b2ba-c4522345b729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:41.025221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:41.065971 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1023/2050] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:41.066774 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f815733-cd83-4339-8228-37c80974d55f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.076003 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f815733-cd83-4339-8228-37c80974d55f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.080730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94821a-8ae0-4350-8373-5c918b311589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.081328 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94821a-8ae0-4350-8373-5c918b311589 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.081609 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94821a-8ae0-4350-8373-5c918b311589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.081874 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d94821a-8ae0-4350-8373-5c918b311589 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.082564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94821a-8ae0-4350-8373-5c918b311589 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:41.082860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94821a-8ae0-4350-8373-5c918b311589 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.083181 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da09be91-980d-4c7e-b2ba-c4522345b729 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.084437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d94821a-8ae0-4350-8373-5c918b311589 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.088090 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a46b27-686d-4afa-b4a3-9e8df445460b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.088694 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a46b27-686d-4afa-b4a3-9e8df445460b admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.088981 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a46b27-686d-4afa-b4a3-9e8df445460b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.089250 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a46b27-686d-4afa-b4a3-9e8df445460b admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.089897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a46b27-686d-4afa-b4a3-9e8df445460b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.092061 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a46b27-686d-4afa-b4a3-9e8df445460b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.101366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1024/2051] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:41.101366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94821a-8ae0-4350-8373-5c918b311589 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.101366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7d94821a-8ae0-4350-8373-5c918b311589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:41.106268 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1028/2052] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:41.108429 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf069a3e-b84c-4c21-b144-64c18e618424 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.108722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf069a3e-b84c-4c21-b144-64c18e618424 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.109677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf069a3e-b84c-4c21-b144-64c18e618424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.112968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d94821a-8ae0-4350-8373-5c918b311589 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.117612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf380a9-e5f1-404b-a66e-ea49304dcb2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.118210 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf380a9-e5f1-404b-a66e-ea49304dcb2e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.118516 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf380a9-e5f1-404b-a66e-ea49304dcb2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.118784 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf380a9-e5f1-404b-a66e-ea49304dcb2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.119405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf380a9-e5f1-404b-a66e-ea49304dcb2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.121292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf380a9-e5f1-404b-a66e-ea49304dcb2e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.143351 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cf069a3e-b84c-4c21-b144-64c18e618424 None None] MFA Rules not processed for user `869317d364c2444693402272266ec73a`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:41.144856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cf069a3e-b84c-4c21-b144-64c18e618424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:41.152601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1025/2053] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:41 2022] POST /identity/v3/projects => generated 367 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:41.156513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dfbd2-85d0-4eea-b771-0bf3f7376c48 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.156513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dfbd2-85d0-4eea-b771-0bf3f7376c48 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.156513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5dfbd2-85d0-4eea-b771-0bf3f7376c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.176298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-eb5dfbd2-85d0-4eea-b771-0bf3f7376c48 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:41.177646 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-eb5dfbd2-85d0-4eea-b771-0bf3f7376c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:41.217720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1029/2054] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:41.218540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf069a3e-b84c-4c21-b144-64c18e618424 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.229387 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf069a3e-b84c-4c21-b144-64c18e618424 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.244117 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1026/2055] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:41.245036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dfbd2-85d0-4eea-b771-0bf3f7376c48 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.254721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5dfbd2-85d0-4eea-b771-0bf3f7376c48 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.259634 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47507d8c-6289-43aa-ae02-f1aaf9e75aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.260236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47507d8c-6289-43aa-ae02-f1aaf9e75aab admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.260515 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47507d8c-6289-43aa-ae02-f1aaf9e75aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.260790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47507d8c-6289-43aa-ae02-f1aaf9e75aab admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.261429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47507d8c-6289-43aa-ae02-f1aaf9e75aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.263059 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47507d8c-6289-43aa-ae02-f1aaf9e75aab admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.268654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:41.274423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1027/2056] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:41.275804 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47507d8c-6289-43aa-ae02-f1aaf9e75aab admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.290109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47507d8c-6289-43aa-ae02-f1aaf9e75aab admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.295334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97541f18-eb95-4e46-8eca-42c081e36e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.295925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97541f18-eb95-4e46-8eca-42c081e36e50 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.296100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704118e-26fa-4708-9def-24c5c2ce46c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.296220 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97541f18-eb95-4e46-8eca-42c081e36e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.296479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97541f18-eb95-4e46-8eca-42c081e36e50 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.296711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704118e-26fa-4708-9def-24c5c2ce46c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.296989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704118e-26fa-4708-9def-24c5c2ce46c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.297099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97541f18-eb95-4e46-8eca-42c081e36e50 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.297253 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f704118e-26fa-4708-9def-24c5c2ce46c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.297887 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f704118e-26fa-4708-9def-24c5c2ce46c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.299127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97541f18-eb95-4e46-8eca-42c081e36e50 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.300324 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f704118e-26fa-4708-9def-24c5c2ce46c6 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.321518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1028/2057] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:41 2022] POST /identity/v3/users => generated 290 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:41.322495 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97541f18-eb95-4e46-8eca-42c081e36e50 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.330034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1030/2058] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:41 2022] POST /identity/v3/projects => generated 381 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:41.331767 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704118e-26fa-4708-9def-24c5c2ce46c6 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.336447 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97541f18-eb95-4e46-8eca-42c081e36e50 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.359635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:41.398331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f704118e-26fa-4708-9def-24c5c2ce46c6 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.403677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908fc75-f58a-4e36-aca2-cc29e9ec899c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.410817 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908fc75-f58a-4e36-aca2-cc29e9ec899c admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.410930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec7877-5993-4c45-aa28-25c4d07ece91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.411051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908fc75-f58a-4e36-aca2-cc29e9ec899c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.411143 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec7877-5993-4c45-aa28-25c4d07ece91 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.411227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1908fc75-f58a-4e36-aca2-cc29e9ec899c admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.411310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec7877-5993-4c45-aa28-25c4d07ece91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.411310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec7877-5993-4c45-aa28-25c4d07ece91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.411466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908fc75-f58a-4e36-aca2-cc29e9ec899c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:41.411551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec7877-5993-4c45-aa28-25c4d07ece91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.411648 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908fc75-f58a-4e36-aca2-cc29e9ec899c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.411737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec7877-5993-4c45-aa28-25c4d07ece91 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.411825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1908fc75-f58a-4e36-aca2-cc29e9ec899c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.426015 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1031/2059] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:41.427050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908fc75-f58a-4e36-aca2-cc29e9ec899c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.442041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1908fc75-f58a-4e36-aca2-cc29e9ec899c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.442442 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1029/2060] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:41 2022] POST /identity/v3/projects => generated 381 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:41.443432 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec7877-5993-4c45-aa28-25c4d07ece91 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.508903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec7877-5993-4c45-aa28-25c4d07ece91 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.509150 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a196f-5228-4ea2-bef8-1a82cb37c0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.509789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a196f-5228-4ea2-bef8-1a82cb37c0a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.510071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a196f-5228-4ea2-bef8-1a82cb37c0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.510378 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22a196f-5228-4ea2-bef8-1a82cb37c0a1 admin admin] PATH_INFO: `/v3/projects/f63eb112769442ddbbe718fd4169cac7/users/23c2101de3874d9ab296aa1e6b5e9b2c/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.515672 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab7330-ebee-4df9-b26f-d8d851cf6138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.516305 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab7330-ebee-4df9-b26f-d8d851cf6138 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.516589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab7330-ebee-4df9-b26f-d8d851cf6138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.516853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab7330-ebee-4df9-b26f-d8d851cf6138 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.517506 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab7330-ebee-4df9-b26f-d8d851cf6138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.519538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab7330-ebee-4df9-b26f-d8d851cf6138 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.526347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a196f-5228-4ea2-bef8-1a82cb37c0a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f63eb112769442ddbbe718fd4169cac7, user_id=23c2101de3874d9ab296aa1e6b5e9b2c, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.530483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c22a196f-5228-4ea2-bef8-1a82cb37c0a1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.535099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1030/2061] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:41.535262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab7330-ebee-4df9-b26f-d8d851cf6138 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.546139 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab7330-ebee-4df9-b26f-d8d851cf6138 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.549439 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1032/2062] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:41 2022] PUT /identity/v3/projects/f63eb112769442ddbbe718fd4169cac7/users/23c2101de3874d9ab296aa1e6b5e9b2c/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:41.550302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a196f-5228-4ea2-bef8-1a82cb37c0a1 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.621903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb44af2-1ab5-478a-aa5b-9611bf86c9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.622502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22a196f-5228-4ea2-bef8-1a82cb37c0a1 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.623370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb44af2-1ab5-478a-aa5b-9611bf86c9b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.623825 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb44af2-1ab5-478a-aa5b-9611bf86c9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.624277 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb44af2-1ab5-478a-aa5b-9611bf86c9b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.625163 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb44af2-1ab5-478a-aa5b-9611bf86c9b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.627500 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb44af2-1ab5-478a-aa5b-9611bf86c9b0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.629971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a0f29-9986-4432-abd0-fafc7ec98b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.631326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a0f29-9986-4432-abd0-fafc7ec98b89 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.631752 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a0f29-9986-4432-abd0-fafc7ec98b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.632044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a0f29-9986-4432-abd0-fafc7ec98b89 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.632730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a0f29-9986-4432-abd0-fafc7ec98b89 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.634951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a0f29-9986-4432-abd0-fafc7ec98b89 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.640498 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:41.659536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1031/2063] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:41 2022] POST /identity/v3/users => generated 291 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:41.660527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb44af2-1ab5-478a-aa5b-9611bf86c9b0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.671991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb44af2-1ab5-478a-aa5b-9611bf86c9b0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.672951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1033/2064] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:41 2022] POST /identity/v3/projects => generated 381 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:41.674068 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a0f29-9986-4432-abd0-fafc7ec98b89 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.704399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:41.743517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a0f29-9986-4432-abd0-fafc7ec98b89 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.748354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb424194-fc11-472c-a105-496f6d54888b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.749012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb424194-fc11-472c-a105-496f6d54888b admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.749295 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb424194-fc11-472c-a105-496f6d54888b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.749691 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb424194-fc11-472c-a105-496f6d54888b admin admin] PATH_INFO: `/v3/projects/891cb927608847608af688ebff79823b/users/e8111ae6212d48739a2fda9265a0f4af/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.751886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b1293-6d36-455c-b875-53a9bf292314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.752839 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b1293-6d36-455c-b875-53a9bf292314 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.753146 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b1293-6d36-455c-b875-53a9bf292314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.753570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b1293-6d36-455c-b875-53a9bf292314 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.754421 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b1293-6d36-455c-b875-53a9bf292314 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.756819 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b1293-6d36-455c-b875-53a9bf292314 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.768682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb424194-fc11-472c-a105-496f6d54888b admin admin] RBAC: Authorizing `identity:create_grant(project_id=891cb927608847608af688ebff79823b, user_id=e8111ae6212d48739a2fda9265a0f4af, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.773360 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb424194-fc11-472c-a105-496f6d54888b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.795943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1032/2065] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:41 2022] POST /identity/v3/users => generated 297 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:41.798203 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1034/2066] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:41 2022] PUT /identity/v3/projects/891cb927608847608af688ebff79823b/users/e8111ae6212d48739a2fda9265a0f4af/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:41.798918 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb424194-fc11-472c-a105-496f6d54888b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.802614 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788425f3-80aa-40d8-bc5a-0b85162123c4 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.803059 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788425f3-80aa-40d8-bc5a-0b85162123c4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.803656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788425f3-80aa-40d8-bc5a-0b85162123c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.849272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-788425f3-80aa-40d8-bc5a-0b85162123c4 None None] MFA Rules not processed for user `23c2101de3874d9ab296aa1e6b5e9b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:41.851584 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-788425f3-80aa-40d8-bc5a-0b85162123c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:41.896795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb424194-fc11-472c-a105-496f6d54888b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.902568 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cff7f-85ea-490e-b474-c69abdd6dd54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.903193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cff7f-85ea-490e-b474-c69abdd6dd54 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.903501 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cff7f-85ea-490e-b474-c69abdd6dd54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.903843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064cff7f-85ea-490e-b474-c69abdd6dd54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.904663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064cff7f-85ea-490e-b474-c69abdd6dd54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.906326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064cff7f-85ea-490e-b474-c69abdd6dd54 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.931442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1035/2067] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:41.931442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cff7f-85ea-490e-b474-c69abdd6dd54 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.947139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064cff7f-85ea-490e-b474-c69abdd6dd54 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.948046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1033/2068] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:41.948786 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788425f3-80aa-40d8-bc5a-0b85162123c4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:41.962597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7708524-a9c7-4f59-aa0e-33ba89a6333d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.963207 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7708524-a9c7-4f59-aa0e-33ba89a6333d admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.963758 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7708524-a9c7-4f59-aa0e-33ba89a6333d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.964316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7708524-a9c7-4f59-aa0e-33ba89a6333d admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.965472 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7708524-a9c7-4f59-aa0e-33ba89a6333d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.967573 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788425f3-80aa-40d8-bc5a-0b85162123c4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:41.969446 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7708524-a9c7-4f59-aa0e-33ba89a6333d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.973727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8dcad-ace6-402a-937c-015412cfc99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:41.974598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8dcad-ace6-402a-937c-015412cfc99b admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:41.974987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8dcad-ace6-402a-937c-015412cfc99b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:41.975795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8dcad-ace6-402a-937c-015412cfc99b admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:41.976344 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8dcad-ace6-402a-937c-015412cfc99b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:41.978528 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8dcad-ace6-402a-937c-015412cfc99b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:41.995913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1034/2069] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:41.997126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8dcad-ace6-402a-937c-015412cfc99b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.016067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8dcad-ace6-402a-937c-015412cfc99b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.022484 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1036/2070] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:41 2022] POST /identity/v3/users => generated 291 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:42.023584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7708524-a9c7-4f59-aa0e-33ba89a6333d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.038845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e1ae7-21a0-40c1-b54c-4c9d651d322c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.039980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e1ae7-21a0-40c1-b54c-4c9d651d322c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.040462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e1ae7-21a0-40c1-b54c-4c9d651d322c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.041232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e1ae7-21a0-40c1-b54c-4c9d651d322c admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.042654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e1ae7-21a0-40c1-b54c-4c9d651d322c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.045328 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4e1ae7-21a0-40c1-b54c-4c9d651d322c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.049525 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7708524-a9c7-4f59-aa0e-33ba89a6333d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.057359 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a252c43-84e4-4ae2-9bf4-a1de4ac8a00c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.058023 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a252c43-84e4-4ae2-9bf4-a1de4ac8a00c admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.058530 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1035/2071] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:41 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:42.059531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e1ae7-21a0-40c1-b54c-4c9d651d322c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.063233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a252c43-84e4-4ae2-9bf4-a1de4ac8a00c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.063233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a252c43-84e4-4ae2-9bf4-a1de4ac8a00c admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.063950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a252c43-84e4-4ae2-9bf4-a1de4ac8a00c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.064698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cf4e1ae7-21a0-40c1-b54c-4c9d651d322c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:42.066719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a252c43-84e4-4ae2-9bf4-a1de4ac8a00c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.087826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1037/2072] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:42.087826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a252c43-84e4-4ae2-9bf4-a1de4ac8a00c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.096689 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a252c43-84e4-4ae2-9bf4-a1de4ac8a00c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.102090 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ec48b-ae76-4d5f-9409-ad8b377f2bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.103283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ec48b-ae76-4d5f-9409-ad8b377f2bda admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.103283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ec48b-ae76-4d5f-9409-ad8b377f2bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.103283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27ec48b-ae76-4d5f-9409-ad8b377f2bda admin admin] PATH_INFO: `/v3/projects/c135997afe334d72bdfd4f8fb570fb07/users/0b83b69675334e6387ff73563415af20/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.123514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ec48b-ae76-4d5f-9409-ad8b377f2bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=c135997afe334d72bdfd4f8fb570fb07, user_id=0b83b69675334e6387ff73563415af20, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.126748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27ec48b-ae76-4d5f-9409-ad8b377f2bda admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.141722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1038/2073] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:42 2022] PUT /identity/v3/projects/c135997afe334d72bdfd4f8fb570fb07/users/0b83b69675334e6387ff73563415af20/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:42.142839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ec48b-ae76-4d5f-9409-ad8b377f2bda admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.150388 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4e1ae7-21a0-40c1-b54c-4c9d651d322c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.154389 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:42.202801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:42.205871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ca26b-47a7-40ee-9a8d-1a729879c3cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.206504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ca26b-47a7-40ee-9a8d-1a729879c3cd service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.206778 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ca26b-47a7-40ee-9a8d-1a729879c3cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.207044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1ca26b-47a7-40ee-9a8d-1a729879c3cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.214957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27ec48b-ae76-4d5f-9409-ad8b377f2bda admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.220724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461c962-617c-4895-83f9-adbd09480865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.221577 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461c962-617c-4895-83f9-adbd09480865 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.222012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461c962-617c-4895-83f9-adbd09480865 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.222403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461c962-617c-4895-83f9-adbd09480865 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.223051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461c962-617c-4895-83f9-adbd09480865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.224650 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461c962-617c-4895-83f9-adbd09480865 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.236015 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1039/2074] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:42.236823 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461c962-617c-4895-83f9-adbd09480865 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.247453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461c962-617c-4895-83f9-adbd09480865 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.253448 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99380b-3081-4357-bcfe-e0df035f8f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.254085 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99380b-3081-4357-bcfe-e0df035f8f28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.254405 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99380b-3081-4357-bcfe-e0df035f8f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.254681 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99380b-3081-4357-bcfe-e0df035f8f28 admin admin] PATH_INFO: `/v3/projects/891cb927608847608af688ebff79823b/users/e8111ae6212d48739a2fda9265a0f4af/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.256598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99380b-3081-4357-bcfe-e0df035f8f28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=891cb927608847608af688ebff79823b, user_id=e8111ae6212d48739a2fda9265a0f4af, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.259303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99380b-3081-4357-bcfe-e0df035f8f28 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.269676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1040/2075] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:42 2022] PUT /identity/v3/projects/891cb927608847608af688ebff79823b/users/e8111ae6212d48739a2fda9265a0f4af/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:42.269896 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ca26b-47a7-40ee-9a8d-1a729879c3cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.270495 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99380b-3081-4357-bcfe-e0df035f8f28 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.271593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1ca26b-47a7-40ee-9a8d-1a729879c3cd service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.285430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1036/2076] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:42.286298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ca26b-47a7-40ee-9a8d-1a729879c3cd service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.299559 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1ca26b-47a7-40ee-9a8d-1a729879c3cd service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.350728 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99380b-3081-4357-bcfe-e0df035f8f28 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.355450 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6fe922-a148-4df2-806e-c96f3432950b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.356579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fe922-a148-4df2-806e-c96f3432950b admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.356579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fe922-a148-4df2-806e-c96f3432950b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.356771 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6fe922-a148-4df2-806e-c96f3432950b admin admin] PATH_INFO: `/v3/projects/904c458b1ecf43cdbd2ee4aaafa6fac1/users/fc7dfbe979b546c58dede0000819938d/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.374851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6fe922-a148-4df2-806e-c96f3432950b admin admin] RBAC: Authorizing `identity:create_grant(project_id=904c458b1ecf43cdbd2ee4aaafa6fac1, user_id=fc7dfbe979b546c58dede0000819938d, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.375904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c410ae-ddf2-43ac-9275-fc20d0dd8ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.376542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c410ae-ddf2-43ac-9275-fc20d0dd8ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.376817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c410ae-ddf2-43ac-9275-fc20d0dd8ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.377091 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c410ae-ddf2-43ac-9275-fc20d0dd8ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.378143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6fe922-a148-4df2-806e-c96f3432950b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.378365 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c410ae-ddf2-43ac-9275-fc20d0dd8ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.379319 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c410ae-ddf2-43ac-9275-fc20d0dd8ab2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.391578 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1037/2077] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:42.392529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c410ae-ddf2-43ac-9275-fc20d0dd8ab2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.400555 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1041/2078] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:42 2022] PUT /identity/v3/projects/904c458b1ecf43cdbd2ee4aaafa6fac1/users/fc7dfbe979b546c58dede0000819938d/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:42.402287 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9efc2-a97f-4cd7-811a-9cada0b886c2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.402760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9efc2-a97f-4cd7-811a-9cada0b886c2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.403032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec9efc2-a97f-4cd7-811a-9cada0b886c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.409037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c410ae-ddf2-43ac-9275-fc20d0dd8ab2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.438133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2ec9efc2-a97f-4cd7-811a-9cada0b886c2 None None] MFA Rules not processed for user `e8111ae6212d48739a2fda9265a0f4af`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:42.439423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2ec9efc2-a97f-4cd7-811a-9cada0b886c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:42.472457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:42.480681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef46b51-7ecc-400f-8448-edd0286b6cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.482730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef46b51-7ecc-400f-8448-edd0286b6cec admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.483103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef46b51-7ecc-400f-8448-edd0286b6cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.483386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef46b51-7ecc-400f-8448-edd0286b6cec admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.484214 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef46b51-7ecc-400f-8448-edd0286b6cec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.485996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef46b51-7ecc-400f-8448-edd0286b6cec admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.506960 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1038/2079] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:42.507723 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef46b51-7ecc-400f-8448-edd0286b6cec admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.518459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef46b51-7ecc-400f-8448-edd0286b6cec admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.524848 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e7d6a-dbcf-4c89-a4ad-f856643bcd2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.530756 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e7d6a-dbcf-4c89-a4ad-f856643bcd2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.531128 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e7d6a-dbcf-4c89-a4ad-f856643bcd2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.531409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e7d6a-dbcf-4c89-a4ad-f856643bcd2a admin admin] PATH_INFO: `/v3/projects/c135997afe334d72bdfd4f8fb570fb07/users/0b83b69675334e6387ff73563415af20/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.550781 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e7d6a-dbcf-4c89-a4ad-f856643bcd2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c135997afe334d72bdfd4f8fb570fb07, user_id=0b83b69675334e6387ff73563415af20, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.550781 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5e7d6a-dbcf-4c89-a4ad-f856643bcd2a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.554798 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1042/2080] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:42 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:42.558073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15039b3-bd14-4d46-b867-662fe0ed52b5 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.558073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15039b3-bd14-4d46-b867-662fe0ed52b5 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.558073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15039b3-bd14-4d46-b867-662fe0ed52b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.561428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1039/2081] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:42 2022] PUT /identity/v3/projects/c135997afe334d72bdfd4f8fb570fb07/users/0b83b69675334e6387ff73563415af20/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:42.561428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e7d6a-dbcf-4c89-a4ad-f856643bcd2a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.599822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f15039b3-bd14-4d46-b867-662fe0ed52b5 None None] MFA Rules not processed for user `fc7dfbe979b546c58dede0000819938d`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:42.601200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f15039b3-bd14-4d46-b867-662fe0ed52b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:42.632684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5e7d6a-dbcf-4c89-a4ad-f856643bcd2a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.638483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294468ee-3aa4-48a2-9e29-1735fcec884f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.639163 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294468ee-3aa4-48a2-9e29-1735fcec884f admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.639611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294468ee-3aa4-48a2-9e29-1735fcec884f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.639897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294468ee-3aa4-48a2-9e29-1735fcec884f admin admin] PATH_INFO: `/v3/projects/00071111c84346639e5245e5c173dad5/users/22c5f040350d4daba0ebe2af86849a23/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.648115 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:42.657840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294468ee-3aa4-48a2-9e29-1735fcec884f admin admin] RBAC: Authorizing `identity:create_grant(project_id=00071111c84346639e5245e5c173dad5, user_id=22c5f040350d4daba0ebe2af86849a23, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.661033 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294468ee-3aa4-48a2-9e29-1735fcec884f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.679855 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1040/2082] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:42 2022] PUT /identity/v3/projects/00071111c84346639e5245e5c173dad5/users/22c5f040350d4daba0ebe2af86849a23/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:42.681652 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f802ae4-cd09-43cd-a85a-12da72eb259a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.682081 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f802ae4-cd09-43cd-a85a-12da72eb259a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.682878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f802ae4-cd09-43cd-a85a-12da72eb259a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.705877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1043/2083] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:42 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:42.707765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15039b3-bd14-4d46-b867-662fe0ed52b5 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.709093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f15039b3-bd14-4d46-b867-662fe0ed52b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:42.716205 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3f802ae4-cd09-43cd-a85a-12da72eb259a None None] MFA Rules not processed for user `0b83b69675334e6387ff73563415af20`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:42.718286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3f802ae4-cd09-43cd-a85a-12da72eb259a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:42.778682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15039b3-bd14-4d46-b867-662fe0ed52b5 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.805337 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1041/2084] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:42 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:42.806829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d19bda-3b8c-40d9-b462-20219b2bb477 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.807108 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d19bda-3b8c-40d9-b462-20219b2bb477 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.807374 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d19bda-3b8c-40d9-b462-20219b2bb477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.834679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b6d19bda-3b8c-40d9-b462-20219b2bb477 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:42.835925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b6d19bda-3b8c-40d9-b462-20219b2bb477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:42.848204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d4932-66e5-4511-ac17-da8b3eca2b85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.848820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d4932-66e5-4511-ac17-da8b3eca2b85 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.849082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d4932-66e5-4511-ac17-da8b3eca2b85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.849344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d4932-66e5-4511-ac17-da8b3eca2b85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.886611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:42.905888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1042/2085] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:42.906858 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d19bda-3b8c-40d9-b462-20219b2bb477 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.916971 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d19bda-3b8c-40d9-b462-20219b2bb477 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.921682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d4932-66e5-4511-ac17-da8b3eca2b85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.922569 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794fe1e-4c7a-4818-9f7d-7c19761a2ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.923174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794fe1e-4c7a-4818-9f7d-7c19761a2ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.923272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d4932-66e5-4511-ac17-da8b3eca2b85 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.923455 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794fe1e-4c7a-4818-9f7d-7c19761a2ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.923920 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794fe1e-4c7a-4818-9f7d-7c19761a2ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.924550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794fe1e-4c7a-4818-9f7d-7c19761a2ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.926496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794fe1e-4c7a-4818-9f7d-7c19761a2ed5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.933857 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1044/2086] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:42 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:42.934831 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d4932-66e5-4511-ac17-da8b3eca2b85 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.944945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d4932-66e5-4511-ac17-da8b3eca2b85 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:42.949365 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41db6b-4ea5-481f-848b-ff35eee77650 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:42.950115 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41db6b-4ea5-481f-848b-ff35eee77650 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:42.951454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41db6b-4ea5-481f-848b-ff35eee77650 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:42.951582 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce41db6b-4ea5-481f-848b-ff35eee77650 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:42.967140 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41db6b-4ea5-481f-848b-ff35eee77650 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:42.968587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce41db6b-4ea5-481f-848b-ff35eee77650 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:42.971523 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1043/2087] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:42 2022] POST /identity/v3/projects => generated 377 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:42.972440 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794fe1e-4c7a-4818-9f7d-7c19761a2ed5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.974158 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6794fe1e-4c7a-4818-9f7d-7c19761a2ed5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:42.985883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1045/2088] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:42 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:42.986890 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41db6b-4ea5-481f-848b-ff35eee77650 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:42.998560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce41db6b-4ea5-481f-848b-ff35eee77650 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:43.025701 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:43.058536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794fe1e-4c7a-4818-9f7d-7c19761a2ed5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:43.071657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d7864-a228-46c6-b26f-4749a89be92e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:43.072525 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d7864-a228-46c6-b26f-4749a89be92e service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:43.072979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d7864-a228-46c6-b26f-4749a89be92e service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:43.073421 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d7864-a228-46c6-b26f-4749a89be92e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:43.088534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830f54b-b398-45fc-9ec4-446249e78609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:43.089153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830f54b-b398-45fc-9ec4-446249e78609 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:43.089434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830f54b-b398-45fc-9ec4-446249e78609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:43.089705 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8830f54b-b398-45fc-9ec4-446249e78609 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:43.090371 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830f54b-b398-45fc-9ec4-446249e78609 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:43.092986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8830f54b-b398-45fc-9ec4-446249e78609 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:43.117041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1046/2089] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:42 2022] POST /identity/v3/users => generated 295 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:43.123890 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830f54b-b398-45fc-9ec4-446249e78609 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:43.141127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8830f54b-b398-45fc-9ec4-446249e78609 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:43.149315 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d4848-d9c1-4791-996c-cb2cd25e596f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:43.150732 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d4848-d9c1-4791-996c-cb2cd25e596f admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:43.150732 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d4848-d9c1-4791-996c-cb2cd25e596f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:43.150930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7d4848-d9c1-4791-996c-cb2cd25e596f admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:43.151721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d4848-d9c1-4791-996c-cb2cd25e596f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:43.153843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7d4848-d9c1-4791-996c-cb2cd25e596f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:43.166576 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1047/2090] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:43.171467 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d4848-d9c1-4791-996c-cb2cd25e596f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:43.177066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d7864-a228-46c6-b26f-4749a89be92e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:43.179089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d7864-a228-46c6-b26f-4749a89be92e service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:43.194054 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7d4848-d9c1-4791-996c-cb2cd25e596f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:43.202046 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b4304-a191-41e5-83ef-1fc1cd6c7ab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:43.203430 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b4304-a191-41e5-83ef-1fc1cd6c7ab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:43.203686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b4304-a191-41e5-83ef-1fc1cd6c7ab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:43.204326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9b4304-a191-41e5-83ef-1fc1cd6c7ab7 admin admin] PATH_INFO: `/v3/projects/389260a6af0a4ae8a83c44a035f5f177/users/5856d09cb22a4371a6b51f20541dd495/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:43.206455 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1044/2091] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:43.233133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b4304-a191-41e5-83ef-1fc1cd6c7ab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=389260a6af0a4ae8a83c44a035f5f177, user_id=5856d09cb22a4371a6b51f20541dd495, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:43.243697 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9b4304-a191-41e5-83ef-1fc1cd6c7ab7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:43.272448 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1048/2092] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:43 2022] PUT /identity/v3/projects/389260a6af0a4ae8a83c44a035f5f177/users/5856d09cb22a4371a6b51f20541dd495/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:43.276593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d7864-a228-46c6-b26f-4749a89be92e service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:43.294641 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d7864-a228-46c6-b26f-4749a89be92e service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:43.400466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4423f84e-8a28-4714-a8d7-82e0e04907f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:43.401210 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423f84e-8a28-4714-a8d7-82e0e04907f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:43.401590 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423f84e-8a28-4714-a8d7-82e0e04907f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:43.402253 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4423f84e-8a28-4714-a8d7-82e0e04907f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:43.403041 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4423f84e-8a28-4714-a8d7-82e0e04907f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:43.404783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4423f84e-8a28-4714-a8d7-82e0e04907f2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:43.420004 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1045/2093] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:43 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:43.424744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b4304-a191-41e5-83ef-1fc1cd6c7ab7 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:43.438743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9b4304-a191-41e5-83ef-1fc1cd6c7ab7 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:43.451306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c5e52-3f11-4d10-aa92-7d35f2adcda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:43.452786 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c5e52-3f11-4d10-aa92-7d35f2adcda1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:43.453093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c5e52-3f11-4d10-aa92-7d35f2adcda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:43.453371 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c5e52-3f11-4d10-aa92-7d35f2adcda1 admin admin] PATH_INFO: `/v3/projects/389260a6af0a4ae8a83c44a035f5f177/users/5856d09cb22a4371a6b51f20541dd495/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:43.461945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c5e52-3f11-4d10-aa92-7d35f2adcda1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=389260a6af0a4ae8a83c44a035f5f177, user_id=5856d09cb22a4371a6b51f20541dd495, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:43.464714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c5e52-3f11-4d10-aa92-7d35f2adcda1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:43.489538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1049/2094] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:43 2022] PUT /identity/v3/projects/389260a6af0a4ae8a83c44a035f5f177/users/5856d09cb22a4371a6b51f20541dd495/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:43.504023 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773e24f-65ca-436d-8880-6cf673a22999 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:43.504804 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773e24f-65ca-436d-8880-6cf673a22999 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:43.505622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773e24f-65ca-436d-8880-6cf673a22999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:43.548197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:43.555894 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0773e24f-65ca-436d-8880-6cf673a22999 None None] MFA Rules not processed for user `5856d09cb22a4371a6b51f20541dd495`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:43.557694 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0773e24f-65ca-436d-8880-6cf673a22999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:43.680480 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1046/2095] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:43.697261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c5e52-3f11-4d10-aa92-7d35f2adcda1 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:43.709752 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773e24f-65ca-436d-8880-6cf673a22999 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:43.772659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c5e52-3f11-4d10-aa92-7d35f2adcda1 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:43.777157 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce50da5-80de-4211-ad3a-e20c43521b51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:43.777792 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce50da5-80de-4211-ad3a-e20c43521b51 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:43.778070 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce50da5-80de-4211-ad3a-e20c43521b51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:43.778455 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce50da5-80de-4211-ad3a-e20c43521b51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:43.790356 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce50da5-80de-4211-ad3a-e20c43521b51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:43.792019 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce50da5-80de-4211-ad3a-e20c43521b51 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:43.803306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1050/2096] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:43.815516 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773e24f-65ca-436d-8880-6cf673a22999 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:43.826158 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621f7e9-0373-43c2-b023-45fc58b25861 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b83b69675334e6387ff73563415af20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c135997afe334d72bdfd4f8fb570fb07', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:43.827181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621f7e9-0373-43c2-b023-45fc58b25861 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:43.827588 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621f7e9-0373-43c2-b023-45fc58b25861 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:43.827972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d621f7e9-0373-43c2-b023-45fc58b25861 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:43.828831 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621f7e9-0373-43c2-b023-45fc58b25861 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:43.829543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621f7e9-0373-43c2-b023-45fc58b25861 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:43.831933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d621f7e9-0373-43c2-b023-45fc58b25861 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:43.858853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1047/2097] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:27:43.862282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce50da5-80de-4211-ad3a-e20c43521b51 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:43.881872 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce50da5-80de-4211-ad3a-e20c43521b51 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:43.889153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af1c164-4253-455e-b8e3-6c9cfd92af03 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b83b69675334e6387ff73563415af20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c135997afe334d72bdfd4f8fb570fb07', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:43.889788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1c164-4253-455e-b8e3-6c9cfd92af03 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:43.890110 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1c164-4253-455e-b8e3-6c9cfd92af03 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:43.890367 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af1c164-4253-455e-b8e3-6c9cfd92af03 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:43.891003 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1c164-4253-455e-b8e3-6c9cfd92af03 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:43.895478 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af1c164-4253-455e-b8e3-6c9cfd92af03 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:43.948666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621f7e9-0373-43c2-b023-45fc58b25861 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:43.958128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1051/2098] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:43 2022] POST /identity/v3/projects => generated 354 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:43.966399 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d621f7e9-0373-43c2-b023-45fc58b25861 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:44.004094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6bd4f-bcd6-4196-b177-5d31b9ed587c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:44.005552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6bd4f-bcd6-4196-b177-5d31b9ed587c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:44.006541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6bd4f-bcd6-4196-b177-5d31b9ed587c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:44.037460 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c2a6bd4f-bcd6-4196-b177-5d31b9ed587c None None] MFA Rules not processed for user `23c2101de3874d9ab296aa1e6b5e9b2c`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:44.039207 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c2a6bd4f-bcd6-4196-b177-5d31b9ed587c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:44.103633 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b64131-b214-4c4b-8e34-221de1743733 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b83b69675334e6387ff73563415af20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c135997afe334d72bdfd4f8fb570fb07', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:44.104760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b64131-b214-4c4b-8e34-221de1743733 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:44.105193 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b64131-b214-4c4b-8e34-221de1743733 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:44.105645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b64131-b214-4c4b-8e34-221de1743733 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:44.106845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b64131-b214-4c4b-8e34-221de1743733 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:44.108980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b64131-b214-4c4b-8e34-221de1743733 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:44.111292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b64131-b214-4c4b-8e34-221de1743733 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:44.135019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1048/2099] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:44.137105 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:44.178449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1052/2100] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:44.208984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b64131-b214-4c4b-8e34-221de1743733 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:44.220354 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b64131-b214-4c4b-8e34-221de1743733 tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:44.227313 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48148704-179c-4e7b-9228-39095ea53efe tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b83b69675334e6387ff73563415af20', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c135997afe334d72bdfd4f8fb570fb07', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:44.228232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48148704-179c-4e7b-9228-39095ea53efe tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:44.228518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48148704-179c-4e7b-9228-39095ea53efe tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:44.228845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48148704-179c-4e7b-9228-39095ea53efe tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] PATH_INFO: `/v3/projects/26494eb5f70c4cc5a38d8954a364ef4a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:44.230192 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48148704-179c-4e7b-9228-39095ea53efe tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] RBAC: Authorizing `identity:delete_project(project_id=26494eb5f70c4cc5a38d8954a364ef4a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:44.232368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48148704-179c-4e7b-9228-39095ea53efe tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:44.271242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:44.352973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48148704-179c-4e7b-9228-39095ea53efe tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26494eb5f70c4cc5a38d8954a364ef4a', 'request_id': 'req-48148704-179c-4e7b-9228-39095ea53efe'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:44.360699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48148704-179c-4e7b-9228-39095ea53efe tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26494eb5f70c4cc5a38d8954a364ef4a', 'request_id': 'req-48148704-179c-4e7b-9228-39095ea53efe'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:44.378851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48148704-179c-4e7b-9228-39095ea53efe tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26494eb5f70c4cc5a38d8954a364ef4a', 'request_id': 'req-48148704-179c-4e7b-9228-39095ea53efe'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:44.410596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48148704-179c-4e7b-9228-39095ea53efe tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] The token cache is being invalidate because project 26494eb5f70c4cc5a38d8954a364ef4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:44.411359 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48148704-179c-4e7b-9228-39095ea53efe tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:44.414063 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48148704-179c-4e7b-9228-39095ea53efe tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:44.420936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1049/2101] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:44 2022] DELETE /identity/v3/projects/26494eb5f70c4cc5a38d8954a364ef4a => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:44.441525 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc00a5-4908-40b9-912e-17e9fde44e7a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:44.442138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc00a5-4908-40b9-912e-17e9fde44e7a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:44.442698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cc00a5-4908-40b9-912e-17e9fde44e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:44.484998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-76cc00a5-4908-40b9-912e-17e9fde44e7a None None] MFA Rules not processed for user `22c5f040350d4daba0ebe2af86849a23`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:44.486497 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-76cc00a5-4908-40b9-912e-17e9fde44e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:44.574847 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1053/2102] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:44.652594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48148704-179c-4e7b-9228-39095ea53efe tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:44.656004 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-48148704-179c-4e7b-9228-39095ea53efe tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:44.759809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48148704-179c-4e7b-9228-39095ea53efe tempest-VolumeQuotaClassesTest-1101686706 tempest-VolumeQuotaClassesTest-1101686706-project-admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:44.765188 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b0d40-3b7f-497c-98a7-035342ae312b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:44.766455 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b0d40-3b7f-497c-98a7-035342ae312b service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:44.766754 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b0d40-3b7f-497c-98a7-035342ae312b service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:44.767017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954b0d40-3b7f-497c-98a7-035342ae312b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:44.781042 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b0d40-3b7f-497c-98a7-035342ae312b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:44.782952 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954b0d40-3b7f-497c-98a7-035342ae312b service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:44.799810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1050/2103] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:44 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:45.068907 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc00a5-4908-40b9-912e-17e9fde44e7a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:45.070932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-76cc00a5-4908-40b9-912e-17e9fde44e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:45.252617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cc00a5-4908-40b9-912e-17e9fde44e7a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:45.259537 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9ba80-faa6-4ff0-8840-ac64f4b219c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:45.261777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9ba80-faa6-4ff0-8840-ac64f4b219c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:45.262717 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9ba80-faa6-4ff0-8840-ac64f4b219c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:45.263193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9ba80-faa6-4ff0-8840-ac64f4b219c0 admin admin] PATH_INFO: `/v3/users/869317d364c2444693402272266ec73a` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:45.265478 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9ba80-faa6-4ff0-8840-ac64f4b219c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=869317d364c2444693402272266ec73a)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:45.270983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9ba80-faa6-4ff0-8840-ac64f4b219c0 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:45.386477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e0d9ba80-faa6-4ff0-8840-ac64f4b219c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '869317d364c2444693402272266ec73a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:45.424709 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e0d9ba80-faa6-4ff0-8840-ac64f4b219c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '869317d364c2444693402272266ec73a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:45.445368 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e0d9ba80-faa6-4ff0-8840-ac64f4b219c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '869317d364c2444693402272266ec73a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:45.446799 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e0d9ba80-faa6-4ff0-8840-ac64f4b219c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '869317d364c2444693402272266ec73a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:45.456989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e0d9ba80-faa6-4ff0-8840-ac64f4b219c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '869317d364c2444693402272266ec73a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:45.460019 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:45.468211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1054/2104] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:45 2022] DELETE /identity/v3/users/869317d364c2444693402272266ec73a => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:45.469571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b0d40-3b7f-497c-98a7-035342ae312b service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:45.472565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-954b0d40-3b7f-497c-98a7-035342ae312b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:45.475204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881a443-6e08-47ed-87fc-05619286e30a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:45.475988 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881a443-6e08-47ed-87fc-05619286e30a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:45.476329 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881a443-6e08-47ed-87fc-05619286e30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:45.502484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954b0d40-3b7f-497c-98a7-035342ae312b service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:45.519772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c881a443-6e08-47ed-87fc-05619286e30a None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:45.521175 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c881a443-6e08-47ed-87fc-05619286e30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:45.610131 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59437c9f-4e74-4214-8b7d-05b5f911234c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:45.610926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59437c9f-4e74-4214-8b7d-05b5f911234c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:45.611245 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59437c9f-4e74-4214-8b7d-05b5f911234c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:45.611482 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59437c9f-4e74-4214-8b7d-05b5f911234c admin admin] PATH_INFO: `/v3/users/fc7dfbe979b546c58dede0000819938d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:45.613477 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59437c9f-4e74-4214-8b7d-05b5f911234c admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc7dfbe979b546c58dede0000819938d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:45.615505 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59437c9f-4e74-4214-8b7d-05b5f911234c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:45.650100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1055/2105] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:45.656117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881a443-6e08-47ed-87fc-05619286e30a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:45.658451 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c881a443-6e08-47ed-87fc-05619286e30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:45.706350 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-59437c9f-4e74-4214-8b7d-05b5f911234c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc7dfbe979b546c58dede0000819938d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:45.706350 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-59437c9f-4e74-4214-8b7d-05b5f911234c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc7dfbe979b546c58dede0000819938d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:45.714847 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:45.729302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-59437c9f-4e74-4214-8b7d-05b5f911234c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc7dfbe979b546c58dede0000819938d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:45.762375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-59437c9f-4e74-4214-8b7d-05b5f911234c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc7dfbe979b546c58dede0000819938d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:45.763771 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-59437c9f-4e74-4214-8b7d-05b5f911234c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc7dfbe979b546c58dede0000819938d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:45.786770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881a443-6e08-47ed-87fc-05619286e30a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:45.791584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349a0fa-bf02-4d9c-adcb-73aa7b35760a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:45.792434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349a0fa-bf02-4d9c-adcb-73aa7b35760a service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:45.792702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349a0fa-bf02-4d9c-adcb-73aa7b35760a service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:45.792976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8349a0fa-bf02-4d9c-adcb-73aa7b35760a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:45.798489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8349a0fa-bf02-4d9c-adcb-73aa7b35760a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:45.805568 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1051/2106] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:45 2022] DELETE /identity/v3/users/fc7dfbe979b546c58dede0000819938d => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:45.815957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f2073-8184-47c0-b4a2-911424ceeb1c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:45.816451 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f2073-8184-47c0-b4a2-911424ceeb1c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:45.817083 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84f2073-8184-47c0-b4a2-911424ceeb1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:45.826818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349a0fa-bf02-4d9c-adcb-73aa7b35760a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:45.828652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8349a0fa-bf02-4d9c-adcb-73aa7b35760a service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:45.847396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1056/2107] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:45.853984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d84f2073-8184-47c0-b4a2-911424ceeb1c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:45.855637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d84f2073-8184-47c0-b4a2-911424ceeb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:45.939979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1052/2108] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:45.955286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349a0fa-bf02-4d9c-adcb-73aa7b35760a service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:45.957318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8349a0fa-bf02-4d9c-adcb-73aa7b35760a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:45.973705 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8349a0fa-bf02-4d9c-adcb-73aa7b35760a service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:46.047390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a96cdf-6ea0-40fb-875f-b1bbc5fa13a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:46.048112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a96cdf-6ea0-40fb-875f-b1bbc5fa13a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:46.048279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a96cdf-6ea0-40fb-875f-b1bbc5fa13a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:46.048555 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a96cdf-6ea0-40fb-875f-b1bbc5fa13a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:46.064937 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a96cdf-6ea0-40fb-875f-b1bbc5fa13a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:46.066882 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a96cdf-6ea0-40fb-875f-b1bbc5fa13a4 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:46.081719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1057/2109] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:46.299529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f2073-8184-47c0-b4a2-911424ceeb1c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:46.299529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d84f2073-8184-47c0-b4a2-911424ceeb1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:46.302309 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a96cdf-6ea0-40fb-875f-b1bbc5fa13a4 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:46.304067 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f9a96cdf-6ea0-40fb-875f-b1bbc5fa13a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:46.310895 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84f2073-8184-47c0-b4a2-911424ceeb1c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:46.316871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cdac93-fa60-45a3-9ca6-95e00d8f7e5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:46.317637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdac93-fa60-45a3-9ca6-95e00d8f7e5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:46.318008 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdac93-fa60-45a3-9ca6-95e00d8f7e5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:46.318621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cdac93-fa60-45a3-9ca6-95e00d8f7e5c admin admin] PATH_INFO: `/v3/users/f8316cc52e4e497f9df463395cd12dd1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:46.319789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdac93-fa60-45a3-9ca6-95e00d8f7e5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8316cc52e4e497f9df463395cd12dd1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:46.321718 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cdac93-fa60-45a3-9ca6-95e00d8f7e5c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:46.323002 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a96cdf-6ea0-40fb-875f-b1bbc5fa13a4 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:46.329963 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d551ada4-9cd8-413a-8659-73326bf48c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:46.333051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551ada4-9cd8-413a-8659-73326bf48c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:46.333051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551ada4-9cd8-413a-8659-73326bf48c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:46.333051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d551ada4-9cd8-413a-8659-73326bf48c50 admin admin] PATH_INFO: `/v3/projects/643f162d549042a6b9b1dc635b1c85e4` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:46.351187 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551ada4-9cd8-413a-8659-73326bf48c50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=643f162d549042a6b9b1dc635b1c85e4)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:46.355613 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d551ada4-9cd8-413a-8659-73326bf48c50 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:46.404824 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-80cdac93-fa60-45a3-9ca6-95e00d8f7e5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8316cc52e4e497f9df463395cd12dd1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.406393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-80cdac93-fa60-45a3-9ca6-95e00d8f7e5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8316cc52e4e497f9df463395cd12dd1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.422142 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:46.428632 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-80cdac93-fa60-45a3-9ca6-95e00d8f7e5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8316cc52e4e497f9df463395cd12dd1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.450254 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d551ada4-9cd8-413a-8659-73326bf48c50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '643f162d549042a6b9b1dc635b1c85e4', 'request_id': 'req-d551ada4-9cd8-413a-8659-73326bf48c50'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.456732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-80cdac93-fa60-45a3-9ca6-95e00d8f7e5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8316cc52e4e497f9df463395cd12dd1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.456732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-80cdac93-fa60-45a3-9ca6-95e00d8f7e5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8316cc52e4e497f9df463395cd12dd1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.461760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d551ada4-9cd8-413a-8659-73326bf48c50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '643f162d549042a6b9b1dc635b1c85e4', 'request_id': 'req-d551ada4-9cd8-413a-8659-73326bf48c50'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.471663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d551ada4-9cd8-413a-8659-73326bf48c50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '643f162d549042a6b9b1dc635b1c85e4', 'request_id': 'req-d551ada4-9cd8-413a-8659-73326bf48c50'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.477230 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1053/2110] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:46 2022] DELETE /identity/v3/users/f8316cc52e4e497f9df463395cd12dd1 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:46.479610 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbeb6b-584f-416a-babe-38d4f4bbd9e3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:46.480268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbeb6b-584f-416a-babe-38d4f4bbd9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:46.482497 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adbeb6b-584f-416a-babe-38d4f4bbd9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:46.490489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d551ada4-9cd8-413a-8659-73326bf48c50 admin admin] The token cache is being invalidate because project 643f162d549042a6b9b1dc635b1c85e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:46.491148 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d551ada4-9cd8-413a-8659-73326bf48c50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.493105 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d551ada4-9cd8-413a-8659-73326bf48c50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.494930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1058/2111] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:46 2022] DELETE /identity/v3/projects/643f162d549042a6b9b1dc635b1c85e4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:46.496898 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b24a5-a303-443a-a7df-1bc4218b4f30 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:46.502758 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b24a5-a303-443a-a7df-1bc4218b4f30 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:46.503036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018b24a5-a303-443a-a7df-1bc4218b4f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:46.509678 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6adbeb6b-584f-416a-babe-38d4f4bbd9e3 None None] MFA Rules not processed for user `22c5f040350d4daba0ebe2af86849a23`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:46.511113 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6adbeb6b-584f-416a-babe-38d4f4bbd9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:46.539166 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-018b24a5-a303-443a-a7df-1bc4218b4f30 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:46.541297 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-018b24a5-a303-443a-a7df-1bc4218b4f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:46.582428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:46.603987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1054/2112] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:46 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:46.604883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbeb6b-584f-416a-babe-38d4f4bbd9e3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:46.606333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6adbeb6b-584f-416a-babe-38d4f4bbd9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:46.614570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1059/2113] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:46.614570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b24a5-a303-443a-a7df-1bc4218b4f30 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:46.616557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-018b24a5-a303-443a-a7df-1bc4218b4f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:46.630779 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018b24a5-a303-443a-a7df-1bc4218b4f30 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:46.638152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee0707-d7c8-40fd-8135-5057209735c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:46.639191 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee0707-d7c8-40fd-8135-5057209735c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:46.639415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adbeb6b-584f-416a-babe-38d4f4bbd9e3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:46.639514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee0707-d7c8-40fd-8135-5057209735c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:46.639863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee0707-d7c8-40fd-8135-5057209735c2 admin admin] PATH_INFO: `/v3/projects/904c458b1ecf43cdbd2ee4aaafa6fac1` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:46.641324 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee0707-d7c8-40fd-8135-5057209735c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=904c458b1ecf43cdbd2ee4aaafa6fac1)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:46.643393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee0707-d7c8-40fd-8135-5057209735c2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:46.645624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1e370-7dd0-4d28-9754-c151384546ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:46.646302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1e370-7dd0-4d28-9754-c151384546ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:46.646582 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1e370-7dd0-4d28-9754-c151384546ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:46.646880 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c1e370-7dd0-4d28-9754-c151384546ad admin admin] PATH_INFO: `/v3/users/0b83b69675334e6387ff73563415af20` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:46.648144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1e370-7dd0-4d28-9754-c151384546ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b83b69675334e6387ff73563415af20)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:46.650174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c1e370-7dd0-4d28-9754-c151384546ad admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:46.727004 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-55c1e370-7dd0-4d28-9754-c151384546ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b83b69675334e6387ff73563415af20'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.727670 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-55c1e370-7dd0-4d28-9754-c151384546ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b83b69675334e6387ff73563415af20'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.728788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c7ee0707-d7c8-40fd-8135-5057209735c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '904c458b1ecf43cdbd2ee4aaafa6fac1', 'request_id': 'req-c7ee0707-d7c8-40fd-8135-5057209735c2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.738837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c7ee0707-d7c8-40fd-8135-5057209735c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '904c458b1ecf43cdbd2ee4aaafa6fac1', 'request_id': 'req-c7ee0707-d7c8-40fd-8135-5057209735c2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.748137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-55c1e370-7dd0-4d28-9754-c151384546ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b83b69675334e6387ff73563415af20'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.762659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c7ee0707-d7c8-40fd-8135-5057209735c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '904c458b1ecf43cdbd2ee4aaafa6fac1', 'request_id': 'req-c7ee0707-d7c8-40fd-8135-5057209735c2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.771037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-55c1e370-7dd0-4d28-9754-c151384546ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b83b69675334e6387ff73563415af20'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.771856 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-55c1e370-7dd0-4d28-9754-c151384546ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b83b69675334e6387ff73563415af20'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.780468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c7ee0707-d7c8-40fd-8135-5057209735c2 admin admin] The token cache is being invalidate because project 904c458b1ecf43cdbd2ee4aaafa6fac1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:46.780468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c7ee0707-d7c8-40fd-8135-5057209735c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.781665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c7ee0707-d7c8-40fd-8135-5057209735c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:46.783136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1060/2114] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:46 2022] DELETE /identity/v3/projects/904c458b1ecf43cdbd2ee4aaafa6fac1 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:46.783996 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee0707-d7c8-40fd-8135-5057209735c2 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:46.785895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c7ee0707-d7c8-40fd-8135-5057209735c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:46.804878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1055/2115] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:46 2022] DELETE /identity/v3/users/0b83b69675334e6387ff73563415af20 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:46.804878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1e370-7dd0-4d28-9754-c151384546ad admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:46.807488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-55c1e370-7dd0-4d28-9754-c151384546ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:46.931011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee0707-d7c8-40fd-8135-5057209735c2 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:46.940751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:46.976967 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c1e370-7dd0-4d28-9754-c151384546ad admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:47.004908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f928a-b690-48e1-af8f-91d3b0e74e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:47.007472 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f928a-b690-48e1-af8f-91d3b0e74e29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:47.008635 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f928a-b690-48e1-af8f-91d3b0e74e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:47.009510 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6f928a-b690-48e1-af8f-91d3b0e74e29 admin admin] PATH_INFO: `/v3/users/5856d09cb22a4371a6b51f20541dd495` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:47.014834 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f928a-b690-48e1-af8f-91d3b0e74e29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5856d09cb22a4371a6b51f20541dd495)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:47.016939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6f928a-b690-48e1-af8f-91d3b0e74e29 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:47.022726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d804d0-0cec-4e60-a97d-782d0a87fda8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:47.023867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d804d0-0cec-4e60-a97d-782d0a87fda8 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:47.024132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d804d0-0cec-4e60-a97d-782d0a87fda8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:47.024400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d804d0-0cec-4e60-a97d-782d0a87fda8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:47.026944 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-37d804d0-0cec-4e60-a97d-782d0a87fda8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:47.063396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d804d0-0cec-4e60-a97d-782d0a87fda8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:47.084530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d804d0-0cec-4e60-a97d-782d0a87fda8 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:47.105059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1061/2116] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:47.132067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e6f928a-b690-48e1-af8f-91d3b0e74e29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5856d09cb22a4371a6b51f20541dd495'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:47.133195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e6f928a-b690-48e1-af8f-91d3b0e74e29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5856d09cb22a4371a6b51f20541dd495'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:47.166044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:47.170464 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e6f928a-b690-48e1-af8f-91d3b0e74e29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5856d09cb22a4371a6b51f20541dd495'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:47.186637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e6f928a-b690-48e1-af8f-91d3b0e74e29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5856d09cb22a4371a6b51f20541dd495'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:47.186903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e6f928a-b690-48e1-af8f-91d3b0e74e29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5856d09cb22a4371a6b51f20541dd495'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:47.208457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1056/2117] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:46 2022] DELETE /identity/v3/users/5856d09cb22a4371a6b51f20541dd495 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:47.612158 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d804d0-0cec-4e60-a97d-782d0a87fda8 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:47.616108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-37d804d0-0cec-4e60-a97d-782d0a87fda8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:47.633379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d804d0-0cec-4e60-a97d-782d0a87fda8 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:47.746257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d8be7d-0a69-4ba4-b989-e223a0c4a12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:47.746920 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8be7d-0a69-4ba4-b989-e223a0c4a12b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:47.747430 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8be7d-0a69-4ba4-b989-e223a0c4a12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:47.747589 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d8be7d-0a69-4ba4-b989-e223a0c4a12b admin admin] PATH_INFO: `/v3/projects/c135997afe334d72bdfd4f8fb570fb07` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:47.748983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8be7d-0a69-4ba4-b989-e223a0c4a12b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c135997afe334d72bdfd4f8fb570fb07)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:47.751527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d8be7d-0a69-4ba4-b989-e223a0c4a12b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:47.805248 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f928a-b690-48e1-af8f-91d3b0e74e29 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:47.806846 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0e6f928a-b690-48e1-af8f-91d3b0e74e29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:47.828967 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6f928a-b690-48e1-af8f-91d3b0e74e29 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:47.832573 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-91d8be7d-0a69-4ba4-b989-e223a0c4a12b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c135997afe334d72bdfd4f8fb570fb07', 'request_id': 'req-91d8be7d-0a69-4ba4-b989-e223a0c4a12b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:47.853330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-91d8be7d-0a69-4ba4-b989-e223a0c4a12b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c135997afe334d72bdfd4f8fb570fb07', 'request_id': 'req-91d8be7d-0a69-4ba4-b989-e223a0c4a12b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:47.862474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-91d8be7d-0a69-4ba4-b989-e223a0c4a12b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c135997afe334d72bdfd4f8fb570fb07', 'request_id': 'req-91d8be7d-0a69-4ba4-b989-e223a0c4a12b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:47.876929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-91d8be7d-0a69-4ba4-b989-e223a0c4a12b admin admin] The token cache is being invalidate because project c135997afe334d72bdfd4f8fb570fb07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:47.877485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-91d8be7d-0a69-4ba4-b989-e223a0c4a12b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:47.878630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-91d8be7d-0a69-4ba4-b989-e223a0c4a12b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:47.881037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1062/2118] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:47 2022] DELETE /identity/v3/projects/c135997afe334d72bdfd4f8fb570fb07 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:47.887172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89083c6-c12f-44b3-b8d9-74274e9b7066 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:47.887592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89083c6-c12f-44b3-b8d9-74274e9b7066 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:47.887969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89083c6-c12f-44b3-b8d9-74274e9b7066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:47.914628 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b89083c6-c12f-44b3-b8d9-74274e9b7066 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:47.916558 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b89083c6-c12f-44b3-b8d9-74274e9b7066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:47.983867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:47.984263 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bd7b5d-21ee-4a2c-8139-680b00c0106d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:47.985978 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd7b5d-21ee-4a2c-8139-680b00c0106d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:47.986290 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd7b5d-21ee-4a2c-8139-680b00c0106d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:47.987111 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bd7b5d-21ee-4a2c-8139-680b00c0106d admin admin] PATH_INFO: `/v3/projects/389260a6af0a4ae8a83c44a035f5f177` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:47.990363 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd7b5d-21ee-4a2c-8139-680b00c0106d admin admin] RBAC: Authorizing `identity:delete_project(project_id=389260a6af0a4ae8a83c44a035f5f177)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:47.992843 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bd7b5d-21ee-4a2c-8139-680b00c0106d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:48.021839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1063/2119] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:48.022518 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89083c6-c12f-44b3-b8d9-74274e9b7066 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:48.024321 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b89083c6-c12f-44b3-b8d9-74274e9b7066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:48.039116 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89083c6-c12f-44b3-b8d9-74274e9b7066 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:48.045788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950172ec-36c9-4ea4-89c6-36258c7f7401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:48.046078 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950172ec-36c9-4ea4-89c6-36258c7f7401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.046780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950172ec-36c9-4ea4-89c6-36258c7f7401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.046780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950172ec-36c9-4ea4-89c6-36258c7f7401 admin admin] PATH_INFO: `/v3/projects/a8401139077d4e009a5597f0ba00e539` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.047998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950172ec-36c9-4ea4-89c6-36258c7f7401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8401139077d4e009a5597f0ba00e539)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:48.050748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950172ec-36c9-4ea4-89c6-36258c7f7401 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:48.065792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48bd7b5d-21ee-4a2c-8139-680b00c0106d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '389260a6af0a4ae8a83c44a035f5f177', 'request_id': 'req-48bd7b5d-21ee-4a2c-8139-680b00c0106d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:48.075054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48bd7b5d-21ee-4a2c-8139-680b00c0106d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '389260a6af0a4ae8a83c44a035f5f177', 'request_id': 'req-48bd7b5d-21ee-4a2c-8139-680b00c0106d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:48.085148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48bd7b5d-21ee-4a2c-8139-680b00c0106d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '389260a6af0a4ae8a83c44a035f5f177', 'request_id': 'req-48bd7b5d-21ee-4a2c-8139-680b00c0106d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:48.099000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48bd7b5d-21ee-4a2c-8139-680b00c0106d admin admin] The token cache is being invalidate because project 389260a6af0a4ae8a83c44a035f5f177 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:48.099341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48bd7b5d-21ee-4a2c-8139-680b00c0106d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:48.100077 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-48bd7b5d-21ee-4a2c-8139-680b00c0106d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:48.101620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1057/2120] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:47 2022] DELETE /identity/v3/projects/389260a6af0a4ae8a83c44a035f5f177 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:48.102873 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898b586-902d-4a57-bca1-3954b247637b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.103149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898b586-902d-4a57-bca1-3954b247637b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.103413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898b586-902d-4a57-bca1-3954b247637b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.109463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-950172ec-36c9-4ea4-89c6-36258c7f7401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8401139077d4e009a5597f0ba00e539', 'request_id': 'req-950172ec-36c9-4ea4-89c6-36258c7f7401'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:48.117910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-950172ec-36c9-4ea4-89c6-36258c7f7401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8401139077d4e009a5597f0ba00e539', 'request_id': 'req-950172ec-36c9-4ea4-89c6-36258c7f7401'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:48.127762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-950172ec-36c9-4ea4-89c6-36258c7f7401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8401139077d4e009a5597f0ba00e539', 'request_id': 'req-950172ec-36c9-4ea4-89c6-36258c7f7401'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:48.128183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9898b586-902d-4a57-bca1-3954b247637b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:48.130895 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9898b586-902d-4a57-bca1-3954b247637b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:48.140337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-950172ec-36c9-4ea4-89c6-36258c7f7401 admin admin] The token cache is being invalidate because project a8401139077d4e009a5597f0ba00e539 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:48.140587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-950172ec-36c9-4ea4-89c6-36258c7f7401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:48.141255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-950172ec-36c9-4ea4-89c6-36258c7f7401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:48.142869 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1064/2121] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:48 2022] DELETE /identity/v3/projects/a8401139077d4e009a5597f0ba00e539 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:48.146048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30927646-bd86-4630-b586-fd760cebaaf1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.146048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30927646-bd86-4630-b586-fd760cebaaf1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.146048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30927646-bd86-4630-b586-fd760cebaaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.165571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-30927646-bd86-4630-b586-fd760cebaaf1 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:48.166849 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-30927646-bd86-4630-b586-fd760cebaaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:48.196966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1058/2122] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:48.199151 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898b586-902d-4a57-bca1-3954b247637b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:48.200736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9898b586-902d-4a57-bca1-3954b247637b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:48.212506 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898b586-902d-4a57-bca1-3954b247637b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:48.217611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fa427-9b9a-40ba-8697-dc485dfc80cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:48.217862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fa427-9b9a-40ba-8697-dc485dfc80cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.218148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fa427-9b9a-40ba-8697-dc485dfc80cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.218459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fa427-9b9a-40ba-8697-dc485dfc80cb admin admin] PATH_INFO: `/v3/users/e8111ae6212d48739a2fda9265a0f4af` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.219602 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48fa427-9b9a-40ba-8697-dc485dfc80cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8111ae6212d48739a2fda9265a0f4af)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:48.221518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48fa427-9b9a-40ba-8697-dc485dfc80cb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:48.226492 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:48.228588 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1065/2123] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:48.229494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30927646-bd86-4630-b586-fd760cebaaf1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:48.239675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30927646-bd86-4630-b586-fd760cebaaf1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:48.244729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77567464-013b-4dae-bfcf-d3d4399c8f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:48.245333 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77567464-013b-4dae-bfcf-d3d4399c8f1d admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.245608 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77567464-013b-4dae-bfcf-d3d4399c8f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.245879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77567464-013b-4dae-bfcf-d3d4399c8f1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.246563 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77567464-013b-4dae-bfcf-d3d4399c8f1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:48.246865 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77567464-013b-4dae-bfcf-d3d4399c8f1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:48.249175 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77567464-013b-4dae-bfcf-d3d4399c8f1d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:48.263776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1066/2124] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:48.265087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80710e98-29c7-4c12-8e5a-6f5527baa284 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.265363 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80710e98-29c7-4c12-8e5a-6f5527baa284 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.265640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80710e98-29c7-4c12-8e5a-6f5527baa284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.279319 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e48fa427-9b9a-40ba-8697-dc485dfc80cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8111ae6212d48739a2fda9265a0f4af'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:48.280011 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e48fa427-9b9a-40ba-8697-dc485dfc80cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8111ae6212d48739a2fda9265a0f4af'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:48.286392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-80710e98-29c7-4c12-8e5a-6f5527baa284 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:48.287564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-80710e98-29c7-4c12-8e5a-6f5527baa284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:48.292603 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e48fa427-9b9a-40ba-8697-dc485dfc80cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8111ae6212d48739a2fda9265a0f4af'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:48.296199 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1067/2125] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:48.297041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80710e98-29c7-4c12-8e5a-6f5527baa284 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:48.304512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e48fa427-9b9a-40ba-8697-dc485dfc80cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8111ae6212d48739a2fda9265a0f4af'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:48.305186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e48fa427-9b9a-40ba-8697-dc485dfc80cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8111ae6212d48739a2fda9265a0f4af'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:48.307359 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80710e98-29c7-4c12-8e5a-6f5527baa284 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:48.312848 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f386e0c-d0e6-4210-adac-f984ea04946d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:48.313455 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f386e0c-d0e6-4210-adac-f984ea04946d admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.313739 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f386e0c-d0e6-4210-adac-f984ea04946d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.314007 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f386e0c-d0e6-4210-adac-f984ea04946d admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.314709 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f386e0c-d0e6-4210-adac-f984ea04946d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:48.316671 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f386e0c-d0e6-4210-adac-f984ea04946d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:48.318336 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1059/2126] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:48 2022] DELETE /identity/v3/users/e8111ae6212d48739a2fda9265a0f4af => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:48.319229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fa427-9b9a-40ba-8697-dc485dfc80cb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:48.320985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e48fa427-9b9a-40ba-8697-dc485dfc80cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:48.364184 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1068/2127] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:48 2022] POST /identity/v3/projects => generated 387 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:48.370690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f386e0c-d0e6-4210-adac-f984ea04946d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:48.377794 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9f386e0c-d0e6-4210-adac-f984ea04946d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:48.418397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:48.432804 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fa427-9b9a-40ba-8697-dc485dfc80cb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:48.475153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f386e0c-d0e6-4210-adac-f984ea04946d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:48.480339 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16418242-3231-4f7b-a1a9-68c497dcee67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:48.480965 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16418242-3231-4f7b-a1a9-68c497dcee67 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.481247 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16418242-3231-4f7b-a1a9-68c497dcee67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.481641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16418242-3231-4f7b-a1a9-68c497dcee67 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.482843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16418242-3231-4f7b-a1a9-68c497dcee67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:48.484801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16418242-3231-4f7b-a1a9-68c497dcee67 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:48.521597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1069/2128] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:48 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:48.526909 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16418242-3231-4f7b-a1a9-68c497dcee67 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:48.533250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a035de0-50bf-46b5-b8e1-664629683fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:48.534195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a035de0-50bf-46b5-b8e1-664629683fc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.534805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a035de0-50bf-46b5-b8e1-664629683fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.535435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a035de0-50bf-46b5-b8e1-664629683fc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.536554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a035de0-50bf-46b5-b8e1-664629683fc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:48.537308 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a035de0-50bf-46b5-b8e1-664629683fc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:48.539951 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a035de0-50bf-46b5-b8e1-664629683fc0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:48.541174 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16418242-3231-4f7b-a1a9-68c497dcee67 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:48.548038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e53917-9caa-4176-8e20-9b65d9055803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:48.548887 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e53917-9caa-4176-8e20-9b65d9055803 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.550519 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e53917-9caa-4176-8e20-9b65d9055803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.550519 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e53917-9caa-4176-8e20-9b65d9055803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.551199 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e53917-9caa-4176-8e20-9b65d9055803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:48.554164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e53917-9caa-4176-8e20-9b65d9055803 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:48.556865 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1060/2129] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:48.569564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a035de0-50bf-46b5-b8e1-664629683fc0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:48.581141 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1070/2130] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:48.584285 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a035de0-50bf-46b5-b8e1-664629683fc0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:48.588525 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e53917-9caa-4176-8e20-9b65d9055803 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:48.604140 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fd2c6-ccdc-4452-83ad-477d13cbd3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:48.604882 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fd2c6-ccdc-4452-83ad-477d13cbd3db admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.605162 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fd2c6-ccdc-4452-83ad-477d13cbd3db admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.605432 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9fd2c6-ccdc-4452-83ad-477d13cbd3db admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.606181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fd2c6-ccdc-4452-83ad-477d13cbd3db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:48.610861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9fd2c6-ccdc-4452-83ad-477d13cbd3db admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:48.613589 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e53917-9caa-4176-8e20-9b65d9055803 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:48.627054 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab6312-5bbe-431c-a99b-10b2d6e0c434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:48.627693 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab6312-5bbe-431c-a99b-10b2d6e0c434 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.627983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab6312-5bbe-431c-a99b-10b2d6e0c434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.628257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dab6312-5bbe-431c-a99b-10b2d6e0c434 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.628912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab6312-5bbe-431c-a99b-10b2d6e0c434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:48.630612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dab6312-5bbe-431c-a99b-10b2d6e0c434 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:48.659448 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1071/2131] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:48.660412 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab6312-5bbe-431c-a99b-10b2d6e0c434 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:48.662060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1061/2132] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:48 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:48.666968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fd2c6-ccdc-4452-83ad-477d13cbd3db admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:48.675449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dab6312-5bbe-431c-a99b-10b2d6e0c434 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:48.681036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:48.760412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9fd2c6-ccdc-4452-83ad-477d13cbd3db admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:48.770702 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f57a7f-6252-4dbb-a31d-bf7a6f8a42a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:48.772307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f57a7f-6252-4dbb-a31d-bf7a6f8a42a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.772307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f57a7f-6252-4dbb-a31d-bf7a6f8a42a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.772307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f57a7f-6252-4dbb-a31d-bf7a6f8a42a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.772677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f57a7f-6252-4dbb-a31d-bf7a6f8a42a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:48.774544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f57a7f-6252-4dbb-a31d-bf7a6f8a42a1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:48.810680 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1062/2133] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:48 2022] POST /identity/v3/users => generated 289 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:48.811952 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f57a7f-6252-4dbb-a31d-bf7a6f8a42a1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:48.823866 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f57a7f-6252-4dbb-a31d-bf7a6f8a42a1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:48.828252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3df445-3865-4d63-8e57-7044b656d052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:48.829002 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3df445-3865-4d63-8e57-7044b656d052 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.829394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3df445-3865-4d63-8e57-7044b656d052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.829675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3df445-3865-4d63-8e57-7044b656d052 admin admin] PATH_INFO: `/v3/projects/6c2c44efca5944f5af0360f82e82c024/users/96c2537762184f0997250bc25422c839/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.830549 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b229b19-873e-4665-8487-d7564f351d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:48.831471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b229b19-873e-4665-8487-d7564f351d87 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.831864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b229b19-873e-4665-8487-d7564f351d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.832176 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b229b19-873e-4665-8487-d7564f351d87 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.832872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b229b19-873e-4665-8487-d7564f351d87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:48.845780 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b229b19-873e-4665-8487-d7564f351d87 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:48.859697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1063/2134] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:48.859986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3df445-3865-4d63-8e57-7044b656d052 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c2c44efca5944f5af0360f82e82c024, user_id=96c2537762184f0997250bc25422c839, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:48.860648 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b229b19-873e-4665-8487-d7564f351d87 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:48.862470 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7b229b19-873e-4665-8487-d7564f351d87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:48.862825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3df445-3865-4d63-8e57-7044b656d052 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:48.878345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b229b19-873e-4665-8487-d7564f351d87 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:48.880213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1072/2135] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:48 2022] PUT /identity/v3/projects/6c2c44efca5944f5af0360f82e82c024/users/96c2537762184f0997250bc25422c839/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:48.881127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3df445-3865-4d63-8e57-7044b656d052 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:48.946414 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291c658-8e7d-40ed-8eca-7746941fe5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:48.947056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291c658-8e7d-40ed-8eca-7746941fe5d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.947339 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291c658-8e7d-40ed-8eca-7746941fe5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.947802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4291c658-8e7d-40ed-8eca-7746941fe5d6 admin admin] PATH_INFO: `/v3/projects/891cb927608847608af688ebff79823b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.948928 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291c658-8e7d-40ed-8eca-7746941fe5d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=891cb927608847608af688ebff79823b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:48.951855 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4291c658-8e7d-40ed-8eca-7746941fe5d6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:48.962094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3df445-3865-4d63-8e57-7044b656d052 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:48.970769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c4a01-a30d-40b1-8e27-195fe9dec268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:48.971424 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c4a01-a30d-40b1-8e27-195fe9dec268 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.971707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c4a01-a30d-40b1-8e27-195fe9dec268 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.971976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c4a01-a30d-40b1-8e27-195fe9dec268 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:48.972809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c4a01-a30d-40b1-8e27-195fe9dec268 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:48.974612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c4a01-a30d-40b1-8e27-195fe9dec268 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:48.987138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1073/2136] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:48.988364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c7a13-247d-4b8a-85d3-7a3fa0f22c7d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:48.988648 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c7a13-247d-4b8a-85d3-7a3fa0f22c7d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:48.988916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c7a13-247d-4b8a-85d3-7a3fa0f22c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.011155 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:49.011366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4291c658-8e7d-40ed-8eca-7746941fe5d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '891cb927608847608af688ebff79823b', 'request_id': 'req-4291c658-8e7d-40ed-8eca-7746941fe5d6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:49.020398 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4291c658-8e7d-40ed-8eca-7746941fe5d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '891cb927608847608af688ebff79823b', 'request_id': 'req-4291c658-8e7d-40ed-8eca-7746941fe5d6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:49.021035 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f36c7a13-247d-4b8a-85d3-7a3fa0f22c7d None None] MFA Rules not processed for user `96c2537762184f0997250bc25422c839`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:49.022342 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f36c7a13-247d-4b8a-85d3-7a3fa0f22c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:49.030080 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4291c658-8e7d-40ed-8eca-7746941fe5d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '891cb927608847608af688ebff79823b', 'request_id': 'req-4291c658-8e7d-40ed-8eca-7746941fe5d6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:49.044517 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4291c658-8e7d-40ed-8eca-7746941fe5d6 admin admin] The token cache is being invalidate because project 891cb927608847608af688ebff79823b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:49.045324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4291c658-8e7d-40ed-8eca-7746941fe5d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:49.046058 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4291c658-8e7d-40ed-8eca-7746941fe5d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:49.048613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1064/2137] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:48 2022] DELETE /identity/v3/projects/891cb927608847608af688ebff79823b => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:49.050081 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291c658-8e7d-40ed-8eca-7746941fe5d6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:49.053227 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4291c658-8e7d-40ed-8eca-7746941fe5d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:49.068150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:49.108376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1074/2138] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:49.110018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad222c2-72d1-4dc7-a6c6-f33b1dfcc914 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.110620 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad222c2-72d1-4dc7-a6c6-f33b1dfcc914 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.110620 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad222c2-72d1-4dc7-a6c6-f33b1dfcc914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.129136 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4291c658-8e7d-40ed-8eca-7746941fe5d6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:49.132183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5ad222c2-72d1-4dc7-a6c6-f33b1dfcc914 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:49.133446 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5ad222c2-72d1-4dc7-a6c6-f33b1dfcc914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:49.136362 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b84cd-6a6d-409a-98ce-76b330fe3954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:49.137044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b84cd-6a6d-409a-98ce-76b330fe3954 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.137320 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b84cd-6a6d-409a-98ce-76b330fe3954 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.137596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b84cd-6a6d-409a-98ce-76b330fe3954 admin admin] PATH_INFO: `/v3/projects/d4b182c4a1b0434c9b0e8c8b7e5fd910/users/5cbc40dd72cf466482f5ba3a400b7e21/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.143082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1075/2139] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:49.144428 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad222c2-72d1-4dc7-a6c6-f33b1dfcc914 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:49.146173 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5ad222c2-72d1-4dc7-a6c6-f33b1dfcc914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:49.164328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad222c2-72d1-4dc7-a6c6-f33b1dfcc914 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:49.165435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b84cd-6a6d-409a-98ce-76b330fe3954 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d4b182c4a1b0434c9b0e8c8b7e5fd910, user_id=5cbc40dd72cf466482f5ba3a400b7e21, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:49.168386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b84cd-6a6d-409a-98ce-76b330fe3954 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:49.169463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c83a4a-f462-45f5-9bd7-dad38fc6569e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:49.170103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c83a4a-f462-45f5-9bd7-dad38fc6569e admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.170419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c83a4a-f462-45f5-9bd7-dad38fc6569e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.170696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c83a4a-f462-45f5-9bd7-dad38fc6569e admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.171324 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c83a4a-f462-45f5-9bd7-dad38fc6569e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:49.173338 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c83a4a-f462-45f5-9bd7-dad38fc6569e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:49.182181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1065/2140] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:49 2022] PUT /identity/v3/projects/d4b182c4a1b0434c9b0e8c8b7e5fd910/users/5cbc40dd72cf466482f5ba3a400b7e21/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:49.188201 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cc75b-9b46-4785-b84d-37460d82a845 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.188461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cc75b-9b46-4785-b84d-37460d82a845 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.188681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4cc75b-9b46-4785-b84d-37460d82a845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.210745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1076/2141] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:49 2022] POST /identity/v3/projects => generated 389 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:49.210745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b6e09-828e-4a5b-adc9-0f482e7d93a2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.210745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b6e09-828e-4a5b-adc9-0f482e7d93a2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.210745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b6e09-828e-4a5b-adc9-0f482e7d93a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.215490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3c4cc75b-9b46-4785-b84d-37460d82a845 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:49.217454 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3c4cc75b-9b46-4785-b84d-37460d82a845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:49.242847 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-af9b6e09-828e-4a5b-adc9-0f482e7d93a2 None None] MFA Rules not processed for user `5cbc40dd72cf466482f5ba3a400b7e21`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:49.245074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-af9b6e09-828e-4a5b-adc9-0f482e7d93a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:49.288386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1066/2142] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:49.289098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4cc75b-9b46-4785-b84d-37460d82a845 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:49.300046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4cc75b-9b46-4785-b84d-37460d82a845 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:49.305632 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1265c-2554-4ca1-ba4e-6c113570e996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:49.306428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1265c-2554-4ca1-ba4e-6c113570e996 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.306716 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1265c-2554-4ca1-ba4e-6c113570e996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.306990 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1265c-2554-4ca1-ba4e-6c113570e996 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.307632 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1265c-2554-4ca1-ba4e-6c113570e996 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:49.309594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af1265c-2554-4ca1-ba4e-6c113570e996 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:49.328470 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1077/2143] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:49 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:49.329707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b6e09-828e-4a5b-adc9-0f482e7d93a2 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:49.339310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1067/2144] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:49 2022] POST /identity/v3/users => generated 301 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:49.340820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9b6e09-828e-4a5b-adc9-0f482e7d93a2 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:49.343235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1265c-2554-4ca1-ba4e-6c113570e996 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:49.346695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc56f91-b614-45f6-bf41-4ede644a4a39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:49.347340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc56f91-b614-45f6-bf41-4ede644a4a39 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.347838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc56f91-b614-45f6-bf41-4ede644a4a39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.348225 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc56f91-b614-45f6-bf41-4ede644a4a39 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.349160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc56f91-b614-45f6-bf41-4ede644a4a39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:49.349468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc56f91-b614-45f6-bf41-4ede644a4a39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:49.351655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc56f91-b614-45f6-bf41-4ede644a4a39 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:49.354556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1265c-2554-4ca1-ba4e-6c113570e996 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:49.363430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6ab53-8738-49f3-802a-c550e2279900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:49.364345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6ab53-8738-49f3-802a-c550e2279900 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.364908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6ab53-8738-49f3-802a-c550e2279900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.365202 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6ab53-8738-49f3-802a-c550e2279900 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.369327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1078/2145] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:49.369481 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6ab53-8738-49f3-802a-c550e2279900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:49.369593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc56f91-b614-45f6-bf41-4ede644a4a39 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:49.369593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8fc56f91-b614-45f6-bf41-4ede644a4a39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:49.369797 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6ab53-8738-49f3-802a-c550e2279900 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:49.387762 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1068/2146] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:49.389254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6ab53-8738-49f3-802a-c550e2279900 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:49.399180 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6ab53-8738-49f3-802a-c550e2279900 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:49.403865 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bfe2c-2822-4804-ad4c-43ee5bd1f49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:49.404455 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bfe2c-2822-4804-ad4c-43ee5bd1f49f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.404740 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bfe2c-2822-4804-ad4c-43ee5bd1f49f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.405012 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0bfe2c-2822-4804-ad4c-43ee5bd1f49f admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.405623 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0bfe2c-2822-4804-ad4c-43ee5bd1f49f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:49.408547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0bfe2c-2822-4804-ad4c-43ee5bd1f49f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:49.432768 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:49.442970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1069/2147] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:49 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:49.442970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bfe2c-2822-4804-ad4c-43ee5bd1f49f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:49.461846 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc56f91-b614-45f6-bf41-4ede644a4a39 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:49.498320 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:49.517346 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0bfe2c-2822-4804-ad4c-43ee5bd1f49f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:49.522382 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a9a02-f7c9-41b7-a2d6-008da56bab59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:49.522532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590da89-888a-4298-a6c6-fb60451e3f21 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96c71ea88d9a474c9ff7a30555cfa91f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:49.523052 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a9a02-f7c9-41b7-a2d6-008da56bab59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.523192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590da89-888a-4298-a6c6-fb60451e3f21 service glance] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.523330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a9a02-f7c9-41b7-a2d6-008da56bab59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.523423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590da89-888a-4298-a6c6-fb60451e3f21 service glance] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.523602 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a9a02-f7c9-41b7-a2d6-008da56bab59 admin admin] PATH_INFO: `/v3/projects/7e9d68f70f304029aa60020c46c608e7/users/69fb37f3d53648029746e13c22cf3229/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.523705 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6590da89-888a-4298-a6c6-fb60451e3f21 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.540544 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a9a02-f7c9-41b7-a2d6-008da56bab59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9d68f70f304029aa60020c46c608e7, user_id=69fb37f3d53648029746e13c22cf3229, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:49.543363 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560a9a02-f7c9-41b7-a2d6-008da56bab59 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:49.557431 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1070/2148] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:49 2022] PUT /identity/v3/projects/7e9d68f70f304029aa60020c46c608e7/users/69fb37f3d53648029746e13c22cf3229/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:49.558263 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a9a02-f7c9-41b7-a2d6-008da56bab59 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:49.592790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6590da89-888a-4298-a6c6-fb60451e3f21 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:49.594274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6590da89-888a-4298-a6c6-fb60451e3f21 service glance] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:49.603804 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1079/2149] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:49 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:49.604605 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590da89-888a-4298-a6c6-fb60451e3f21 service glance] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:49.615663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6590da89-888a-4298-a6c6-fb60451e3f21 service glance] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:49.621453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9df050-e486-4c21-9c46-2cc5764a2aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:49.622269 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9df050-e486-4c21-9c46-2cc5764a2aad admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.623013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9df050-e486-4c21-9c46-2cc5764a2aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.623282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9df050-e486-4c21-9c46-2cc5764a2aad admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.623928 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9df050-e486-4c21-9c46-2cc5764a2aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:49.625508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9df050-e486-4c21-9c46-2cc5764a2aad admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:49.626708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560a9a02-f7c9-41b7-a2d6-008da56bab59 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:49.633262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d1020-f11e-4fdf-8357-932a7c56c576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:49.633965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d1020-f11e-4fdf-8357-932a7c56c576 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.634288 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d1020-f11e-4fdf-8357-932a7c56c576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.634624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7d1020-f11e-4fdf-8357-932a7c56c576 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.635357 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d1020-f11e-4fdf-8357-932a7c56c576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:49.637321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7d1020-f11e-4fdf-8357-932a7c56c576 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:49.646652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1080/2150] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:49.647903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9df050-e486-4c21-9c46-2cc5764a2aad admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:49.668111 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9df050-e486-4c21-9c46-2cc5764a2aad admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:49.671619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1071/2151] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:49 2022] POST /identity/v3/users => generated 283 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:49.675882 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d1020-f11e-4fdf-8357-932a7c56c576 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:49.678160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3fcb8-a71c-4afa-99dd-be4a7c9092f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:49.679044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3fcb8-a71c-4afa-99dd-be4a7c9092f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.679380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3fcb8-a71c-4afa-99dd-be4a7c9092f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.679723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3fcb8-a71c-4afa-99dd-be4a7c9092f5 admin admin] PATH_INFO: `/v3/projects/7e9d68f70f304029aa60020c46c608e7/users/69fb37f3d53648029746e13c22cf3229/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.682599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3fcb8-a71c-4afa-99dd-be4a7c9092f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e9d68f70f304029aa60020c46c608e7, user_id=69fb37f3d53648029746e13c22cf3229, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:49.687478 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7d1020-f11e-4fdf-8357-932a7c56c576 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:49.688218 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3fcb8-a71c-4afa-99dd-be4a7c9092f5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:49.699776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266511f-606a-4ee8-94b1-756b58d1cd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:49.700399 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266511f-606a-4ee8-94b1-756b58d1cd43 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.701230 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266511f-606a-4ee8-94b1-756b58d1cd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.701505 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266511f-606a-4ee8-94b1-756b58d1cd43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.702270 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266511f-606a-4ee8-94b1-756b58d1cd43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:49.703888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266511f-606a-4ee8-94b1-756b58d1cd43 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:49.714277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1081/2152] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:49 2022] PUT /identity/v3/projects/7e9d68f70f304029aa60020c46c608e7/users/69fb37f3d53648029746e13c22cf3229/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:49.719565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1072/2153] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:49.721623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c351735-68d2-4792-921a-a3d80d87831f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.722395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c351735-68d2-4792-921a-a3d80d87831f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.722704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c351735-68d2-4792-921a-a3d80d87831f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.724994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266511f-606a-4ee8-94b1-756b58d1cd43 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:49.748368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266511f-606a-4ee8-94b1-756b58d1cd43 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:49.792726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3c351735-68d2-4792-921a-a3d80d87831f None None] MFA Rules not processed for user `69fb37f3d53648029746e13c22cf3229`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:49.794505 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3c351735-68d2-4792-921a-a3d80d87831f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:49.850726 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e659ce-0c1a-4c2f-9082-df89e3a172dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:49.851421 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e659ce-0c1a-4c2f-9082-df89e3a172dd admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.851712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e659ce-0c1a-4c2f-9082-df89e3a172dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.851983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e659ce-0c1a-4c2f-9082-df89e3a172dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.852636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e659ce-0c1a-4c2f-9082-df89e3a172dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:49.855321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e659ce-0c1a-4c2f-9082-df89e3a172dd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:49.901141 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1073/2154] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:49 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:49.901621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:49.901897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e659ce-0c1a-4c2f-9082-df89e3a172dd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:49.908653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1082/2155] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:49 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:49.925543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e659ce-0c1a-4c2f-9082-df89e3a172dd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:49.935035 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c351735-68d2-4792-921a-a3d80d87831f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:49.935798 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989d7d37-b18b-4d51-8ff5-18f146057732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:49.936599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3c351735-68d2-4792-921a-a3d80d87831f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:49.937233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d7d37-b18b-4d51-8ff5-18f146057732 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:49.937818 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d7d37-b18b-4d51-8ff5-18f146057732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:49.938434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989d7d37-b18b-4d51-8ff5-18f146057732 admin admin] PATH_INFO: `/v3/projects/c6f2a1c9f08443c3a527c618c2a895fb/users/f21bd5d9b09847cd9a19ebc1c80da10a/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:49.980561 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d7d37-b18b-4d51-8ff5-18f146057732 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6f2a1c9f08443c3a527c618c2a895fb, user_id=f21bd5d9b09847cd9a19ebc1c80da10a, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:49.996110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989d7d37-b18b-4d51-8ff5-18f146057732 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:50.016242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1074/2156] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:49 2022] PUT /identity/v3/projects/c6f2a1c9f08443c3a527c618c2a895fb/users/f21bd5d9b09847cd9a19ebc1c80da10a/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:50.021695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c351735-68d2-4792-921a-a3d80d87831f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:50.030459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68e3c7-74d6-4190-8bc9-155079aa4af2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.030810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68e3c7-74d6-4190-8bc9-155079aa4af2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.031154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68e3c7-74d6-4190-8bc9-155079aa4af2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.070411 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7e68e3c7-74d6-4190-8bc9-155079aa4af2 None None] MFA Rules not processed for user `f21bd5d9b09847cd9a19ebc1c80da10a`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:50.071874 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7e68e3c7-74d6-4190-8bc9-155079aa4af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:50.078578 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:50.083811 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6cafc6-703c-4aaf-8d8b-53264ffb249b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:50.084477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cafc6-703c-4aaf-8d8b-53264ffb249b service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.084735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cafc6-703c-4aaf-8d8b-53264ffb249b service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.085167 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6cafc6-703c-4aaf-8d8b-53264ffb249b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.158830 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6cafc6-703c-4aaf-8d8b-53264ffb249b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:50.160304 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6cafc6-703c-4aaf-8d8b-53264ffb249b service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:50.172491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1083/2157] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:49 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:50.204565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1075/2158] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:50.206006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6cafc6-703c-4aaf-8d8b-53264ffb249b service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:50.295665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6cafc6-703c-4aaf-8d8b-53264ffb249b service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:50.301750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc59ad0-0e7b-405c-b1ce-08461da8d7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:50.302425 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc59ad0-0e7b-405c-b1ce-08461da8d7e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.302709 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc59ad0-0e7b-405c-b1ce-08461da8d7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.302976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc59ad0-0e7b-405c-b1ce-08461da8d7e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.303927 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc59ad0-0e7b-405c-b1ce-08461da8d7e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:50.305898 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc59ad0-0e7b-405c-b1ce-08461da8d7e8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:50.320545 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:50.349744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1084/2159] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:50 2022] POST /identity/v3/projects => generated 367 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:50.350964 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68e3c7-74d6-4190-8bc9-155079aa4af2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:50.362131 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68e3c7-74d6-4190-8bc9-155079aa4af2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:50.366776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc59ad0-0e7b-405c-b1ce-08461da8d7e8 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:50.423046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113b553-a89a-4c09-ae9a-bde040d62e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:50.423761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113b553-a89a-4c09-ae9a-bde040d62e3f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.424113 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113b553-a89a-4c09-ae9a-bde040d62e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.424457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d113b553-a89a-4c09-ae9a-bde040d62e3f admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.425091 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113b553-a89a-4c09-ae9a-bde040d62e3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:50.427033 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d113b553-a89a-4c09-ae9a-bde040d62e3f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:50.433048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc59ad0-0e7b-405c-b1ce-08461da8d7e8 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:50.439480 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c5c4b-1a97-4b7a-a692-c82796927050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:50.440198 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c5c4b-1a97-4b7a-a692-c82796927050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.440479 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c5c4b-1a97-4b7a-a692-c82796927050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.440740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c5c4b-1a97-4b7a-a692-c82796927050 admin admin] PATH_INFO: `/v3/users/96c2537762184f0997250bc25422c839` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.441913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c5c4b-1a97-4b7a-a692-c82796927050 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96c2537762184f0997250bc25422c839)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:50.443895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672c5c4b-1a97-4b7a-a692-c82796927050 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:50.449421 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1076/2160] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:50 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:50.453351 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113b553-a89a-4c09-ae9a-bde040d62e3f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:50.463342 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d113b553-a89a-4c09-ae9a-bde040d62e3f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:50.468699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eecf72-031d-4924-84ed-edc208192f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:50.469313 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eecf72-031d-4924-84ed-edc208192f8c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.469589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eecf72-031d-4924-84ed-edc208192f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.469870 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eecf72-031d-4924-84ed-edc208192f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.470566 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eecf72-031d-4924-84ed-edc208192f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:50.472173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eecf72-031d-4924-84ed-edc208192f8c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:50.484503 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1077/2161] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:50.489853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eecf72-031d-4924-84ed-edc208192f8c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:50.502131 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eecf72-031d-4924-84ed-edc208192f8c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:50.505346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-672c5c4b-1a97-4b7a-a692-c82796927050 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96c2537762184f0997250bc25422c839'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:50.511566 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11266df0-87ac-4399-b6f8-0a7c1a4118a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:50.512333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11266df0-87ac-4399-b6f8-0a7c1a4118a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.512682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11266df0-87ac-4399-b6f8-0a7c1a4118a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.512874 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11266df0-87ac-4399-b6f8-0a7c1a4118a1 admin admin] PATH_INFO: `/v3/projects/fd735a58080d46d480bc84e8e24fd051/users/8fb6c86a7b0340d0b48afef18d00e221/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.514372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-672c5c4b-1a97-4b7a-a692-c82796927050 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96c2537762184f0997250bc25422c839'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:50.529812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11266df0-87ac-4399-b6f8-0a7c1a4118a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd735a58080d46d480bc84e8e24fd051, user_id=8fb6c86a7b0340d0b48afef18d00e221, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:50.530042 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-672c5c4b-1a97-4b7a-a692-c82796927050 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96c2537762184f0997250bc25422c839'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:50.530685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-672c5c4b-1a97-4b7a-a692-c82796927050 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96c2537762184f0997250bc25422c839'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:50.532931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11266df0-87ac-4399-b6f8-0a7c1a4118a1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:50.538313 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-672c5c4b-1a97-4b7a-a692-c82796927050 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96c2537762184f0997250bc25422c839'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:50.546083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1085/2162] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:50 2022] DELETE /identity/v3/users/96c2537762184f0997250bc25422c839 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:50.549537 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1078/2163] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:50 2022] PUT /identity/v3/projects/fd735a58080d46d480bc84e8e24fd051/users/8fb6c86a7b0340d0b48afef18d00e221/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:50.550771 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0aae7-4670-4fde-9a09-fbe829c75994 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.551006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c5c4b-1a97-4b7a-a692-c82796927050 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:50.551116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0aae7-4670-4fde-9a09-fbe829c75994 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.551335 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d0aae7-4670-4fde-9a09-fbe829c75994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.552445 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-672c5c4b-1a97-4b7a-a692-c82796927050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:50.572076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c4d0aae7-4670-4fde-9a09-fbe829c75994 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:50.573252 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c4d0aae7-4670-4fde-9a09-fbe829c75994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:50.622064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c5c4b-1a97-4b7a-a692-c82796927050 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:50.627835 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0d4ab-bfe8-4a34-a342-1401a6849314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:50.628450 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0d4ab-bfe8-4a34-a342-1401a6849314 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.628733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0d4ab-bfe8-4a34-a342-1401a6849314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.629004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0d4ab-bfe8-4a34-a342-1401a6849314 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.629652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a0d4ab-bfe8-4a34-a342-1401a6849314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:50.631324 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a0d4ab-bfe8-4a34-a342-1401a6849314 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:50.642039 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1086/2164] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:50.644486 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1079/2165] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:50.646279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0d4ab-bfe8-4a34-a342-1401a6849314 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:50.654184 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d0aae7-4670-4fde-9a09-fbe829c75994 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:50.655665 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c4d0aae7-4670-4fde-9a09-fbe829c75994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:50.656601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0d4ab-bfe8-4a34-a342-1401a6849314 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:50.661108 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:50.661855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99268be-b418-4d65-84fb-89ec70272c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:50.662172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99268be-b418-4d65-84fb-89ec70272c5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.662597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99268be-b418-4d65-84fb-89ec70272c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.663228 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99268be-b418-4d65-84fb-89ec70272c5d admin admin] PATH_INFO: `/v3/projects/fd735a58080d46d480bc84e8e24fd051/users/8fb6c86a7b0340d0b48afef18d00e221/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.664930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99268be-b418-4d65-84fb-89ec70272c5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd735a58080d46d480bc84e8e24fd051, user_id=8fb6c86a7b0340d0b48afef18d00e221, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:50.667641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99268be-b418-4d65-84fb-89ec70272c5d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:50.670738 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:50.682947 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1087/2166] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:50 2022] PUT /identity/v3/projects/fd735a58080d46d480bc84e8e24fd051/users/8fb6c86a7b0340d0b48afef18d00e221/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:50.694523 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017319e4-9578-45de-95fd-c99b130b2973 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.694815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017319e4-9578-45de-95fd-c99b130b2973 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.695113 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017319e4-9578-45de-95fd-c99b130b2973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.719304 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d0aae7-4670-4fde-9a09-fbe829c75994 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:50.724116 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-017319e4-9578-45de-95fd-c99b130b2973 None None] MFA Rules not processed for user `8fb6c86a7b0340d0b48afef18d00e221`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:50.725331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-017319e4-9578-45de-95fd-c99b130b2973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:50.772198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0305d22-b704-42f6-b248-c1479b757fd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:50.772909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0305d22-b704-42f6-b248-c1479b757fd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.773186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0305d22-b704-42f6-b248-c1479b757fd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.773459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0305d22-b704-42f6-b248-c1479b757fd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.794960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1088/2167] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:50.807145 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017319e4-9578-45de-95fd-c99b130b2973 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:50.808842 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-017319e4-9578-45de-95fd-c99b130b2973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:50.836808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0305d22-b704-42f6-b248-c1479b757fd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:50.838311 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0305d22-b704-42f6-b248-c1479b757fd6 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:50.858640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1080/2168] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:50.869531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017319e4-9578-45de-95fd-c99b130b2973 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:50.874748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6a013-8da1-4685-b72e-58699d4ddc0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:50.875520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6a013-8da1-4685-b72e-58699d4ddc0c service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.875862 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6a013-8da1-4685-b72e-58699d4ddc0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.876141 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af6a013-8da1-4685-b72e-58699d4ddc0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.887611 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6a013-8da1-4685-b72e-58699d4ddc0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:50.890397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af6a013-8da1-4685-b72e-58699d4ddc0c service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:50.901282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1089/2169] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:50 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:50.952659 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0305d22-b704-42f6-b248-c1479b757fd6 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:50.954010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c0305d22-b704-42f6-b248-c1479b757fd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:50.967799 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0305d22-b704-42f6-b248-c1479b757fd6 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:50.972608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa7be1-3b2a-4a32-a3cb-ba0128875b0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:50.973155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa7be1-3b2a-4a32-a3cb-ba0128875b0a service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:50.973436 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa7be1-3b2a-4a32-a3cb-ba0128875b0a service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:50.973709 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfa7be1-3b2a-4a32-a3cb-ba0128875b0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:50.979083 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1dfa7be1-3b2a-4a32-a3cb-ba0128875b0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:51.098757 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa7be1-3b2a-4a32-a3cb-ba0128875b0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:51.100488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfa7be1-3b2a-4a32-a3cb-ba0128875b0a service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:51.113695 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1081/2170] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:51.294913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6a013-8da1-4685-b72e-58699d4ddc0c service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:51.296989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9af6a013-8da1-4685-b72e-58699d4ddc0c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:51.327553 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af6a013-8da1-4685-b72e-58699d4ddc0c service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:51.333077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:51.333732 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:51.334004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:51.334314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6 admin admin] PATH_INFO: `/v3/projects/6c2c44efca5944f5af0360f82e82c024` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:51.335488 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c2c44efca5944f5af0360f82e82c024)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:51.337407 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:51.413080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c2c44efca5944f5af0360f82e82c024', 'request_id': 'req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:51.421896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c2c44efca5944f5af0360f82e82c024', 'request_id': 'req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:51.431313 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c2c44efca5944f5af0360f82e82c024', 'request_id': 'req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:51.444574 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6 admin admin] The token cache is being invalidate because project 6c2c44efca5944f5af0360f82e82c024 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:51.444902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:51.445623 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:51.450018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1090/2171] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:51 2022] DELETE /identity/v3/projects/6c2c44efca5944f5af0360f82e82c024 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:51.451307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa7be1-3b2a-4a32-a3cb-ba0128875b0a service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:51.462861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfa7be1-3b2a-4a32-a3cb-ba0128875b0a service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:51.466938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:51.537598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dc5126-6d31-4d9c-bfb7-c8c1178ad428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:51.538687 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc5126-6d31-4d9c-bfb7-c8c1178ad428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:51.539156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc5126-6d31-4d9c-bfb7-c8c1178ad428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:51.539862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dc5126-6d31-4d9c-bfb7-c8c1178ad428 admin admin] PATH_INFO: `/v3/users/69fb37f3d53648029746e13c22cf3229` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:51.541784 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc5126-6d31-4d9c-bfb7-c8c1178ad428 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69fb37f3d53648029746e13c22cf3229)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:51.545837 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dc5126-6d31-4d9c-bfb7-c8c1178ad428 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:51.600927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-45dc5126-6d31-4d9c-bfb7-c8c1178ad428 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69fb37f3d53648029746e13c22cf3229'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:51.601569 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-45dc5126-6d31-4d9c-bfb7-c8c1178ad428 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69fb37f3d53648029746e13c22cf3229'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:51.614105 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-45dc5126-6d31-4d9c-bfb7-c8c1178ad428 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69fb37f3d53648029746e13c22cf3229'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:51.621625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-45dc5126-6d31-4d9c-bfb7-c8c1178ad428 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69fb37f3d53648029746e13c22cf3229'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:51.622276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-45dc5126-6d31-4d9c-bfb7-c8c1178ad428 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69fb37f3d53648029746e13c22cf3229'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:51.634626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1082/2172] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:51 2022] DELETE /identity/v3/users/69fb37f3d53648029746e13c22cf3229 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:51.909817 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:51.911318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:51.923941 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:51.928526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c79ddf-dc85-4ca6-866d-af4b90c737d6 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:51.996602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394409ef-6dec-4be6-8724-adb18d324863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:51.998060 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394409ef-6dec-4be6-8724-adb18d324863 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:51.998060 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394409ef-6dec-4be6-8724-adb18d324863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:51.998060 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394409ef-6dec-4be6-8724-adb18d324863 admin admin] PATH_INFO: `/v3/projects/7e9d68f70f304029aa60020c46c608e7` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:51.999587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394409ef-6dec-4be6-8724-adb18d324863 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e9d68f70f304029aa60020c46c608e7)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:52.001773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394409ef-6dec-4be6-8724-adb18d324863 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:52.089506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-394409ef-6dec-4be6-8724-adb18d324863 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e9d68f70f304029aa60020c46c608e7', 'request_id': 'req-394409ef-6dec-4be6-8724-adb18d324863'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:52.097288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-394409ef-6dec-4be6-8724-adb18d324863 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e9d68f70f304029aa60020c46c608e7', 'request_id': 'req-394409ef-6dec-4be6-8724-adb18d324863'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:52.105380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-394409ef-6dec-4be6-8724-adb18d324863 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e9d68f70f304029aa60020c46c608e7', 'request_id': 'req-394409ef-6dec-4be6-8724-adb18d324863'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:52.116527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-394409ef-6dec-4be6-8724-adb18d324863 admin admin] The token cache is being invalidate because project 7e9d68f70f304029aa60020c46c608e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:52.116813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-394409ef-6dec-4be6-8724-adb18d324863 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:52.117454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-394409ef-6dec-4be6-8724-adb18d324863 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:52.118989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1091/2173] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:51 2022] DELETE /identity/v3/projects/7e9d68f70f304029aa60020c46c608e7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:52.125009 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fa64f-8d35-4c02-8401-794606f431ee None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:52.125521 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fa64f-8d35-4c02-8401-794606f431ee None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:52.125616 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245fa64f-8d35-4c02-8401-794606f431ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:52.148821 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-245fa64f-8d35-4c02-8401-794606f431ee None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:52.150040 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-245fa64f-8d35-4c02-8401-794606f431ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:52.219894 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1083/2174] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:52.229824 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31106430-f05a-4414-9783-8700949b8699 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:52.230417 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31106430-f05a-4414-9783-8700949b8699 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:52.230758 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31106430-f05a-4414-9783-8700949b8699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:52.255314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-31106430-f05a-4414-9783-8700949b8699 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:52.256393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-31106430-f05a-4414-9783-8700949b8699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:52.265156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1092/2175] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:52.271613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fa64f-8d35-4c02-8401-794606f431ee None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:52.287284 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245fa64f-8d35-4c02-8401-794606f431ee None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:52.295044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a6fc2-f352-4171-9ac2-24616936e704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:52.295823 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a6fc2-f352-4171-9ac2-24616936e704 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:52.296106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a6fc2-f352-4171-9ac2-24616936e704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:52.296397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3a6fc2-f352-4171-9ac2-24616936e704 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:52.297253 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a6fc2-f352-4171-9ac2-24616936e704 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:52.297550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a6fc2-f352-4171-9ac2-24616936e704 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:52.299661 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3a6fc2-f352-4171-9ac2-24616936e704 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:52.318269 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1084/2176] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:52.322432 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31106430-f05a-4414-9783-8700949b8699 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:52.335897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31106430-f05a-4414-9783-8700949b8699 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:52.341910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e391c5-122a-46b4-af55-cef1c0a147ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:52.342510 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e391c5-122a-46b4-af55-cef1c0a147ac admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:52.342841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e391c5-122a-46b4-af55-cef1c0a147ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:52.343128 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e391c5-122a-46b4-af55-cef1c0a147ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:52.344189 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e391c5-122a-46b4-af55-cef1c0a147ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:52.361801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e391c5-122a-46b4-af55-cef1c0a147ac admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:52.433265 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1093/2177] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:52 2022] POST /identity/v3/projects => generated 383 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:52.439142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a6fc2-f352-4171-9ac2-24616936e704 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:52.455511 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3a6fc2-f352-4171-9ac2-24616936e704 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:52.519487 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:52.637593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280e102-a6e1-4f90-b412-63a6ee2ef89b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:52.650256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280e102-a6e1-4f90-b412-63a6ee2ef89b admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:52.650256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280e102-a6e1-4f90-b412-63a6ee2ef89b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:52.650256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a280e102-a6e1-4f90-b412-63a6ee2ef89b admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:52.650256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280e102-a6e1-4f90-b412-63a6ee2ef89b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:52.650256 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a280e102-a6e1-4f90-b412-63a6ee2ef89b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:52.677176 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1085/2178] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:52 2022] POST /identity/v3/users => generated 292 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:52.682365 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e391c5-122a-46b4-af55-cef1c0a147ac admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:52.701664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e391c5-122a-46b4-af55-cef1c0a147ac admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:52.719968 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d18796-8a52-4a19-950e-bc4ee4f75e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:52.720695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d18796-8a52-4a19-950e-bc4ee4f75e3e admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:52.721067 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d18796-8a52-4a19-950e-bc4ee4f75e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:52.721491 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d18796-8a52-4a19-950e-bc4ee4f75e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:52.723382 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d18796-8a52-4a19-950e-bc4ee4f75e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:52.725481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d18796-8a52-4a19-950e-bc4ee4f75e3e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:52.745522 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1094/2179] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:52.746942 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280e102-a6e1-4f90-b412-63a6ee2ef89b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:52.761873 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a280e102-a6e1-4f90-b412-63a6ee2ef89b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:52.768008 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e9d86c-9281-4180-92b8-22faabf1a0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:52.768880 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9d86c-9281-4180-92b8-22faabf1a0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:52.769145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9d86c-9281-4180-92b8-22faabf1a0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:52.769467 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e9d86c-9281-4180-92b8-22faabf1a0e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:52.770099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9d86c-9281-4180-92b8-22faabf1a0e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:52.773340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e9d86c-9281-4180-92b8-22faabf1a0e4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:52.787707 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1086/2180] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:52.794720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d18796-8a52-4a19-950e-bc4ee4f75e3e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:52.811282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d18796-8a52-4a19-950e-bc4ee4f75e3e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:52.817987 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963c4c1-09b3-4a43-96d4-0212c4b36bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:52.818917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963c4c1-09b3-4a43-96d4-0212c4b36bf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:52.819302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963c4c1-09b3-4a43-96d4-0212c4b36bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:52.819610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963c4c1-09b3-4a43-96d4-0212c4b36bf7 admin admin] PATH_INFO: `/v3/projects/f08b45020e27401bb97912ac9e1f15bd/users/1020903e2d434f15bf623bcf3b7d2202/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:52.847341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963c4c1-09b3-4a43-96d4-0212c4b36bf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f08b45020e27401bb97912ac9e1f15bd, user_id=1020903e2d434f15bf623bcf3b7d2202, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:52.851015 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963c4c1-09b3-4a43-96d4-0212c4b36bf7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:52.874452 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1095/2181] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:52 2022] PUT /identity/v3/projects/f08b45020e27401bb97912ac9e1f15bd/users/1020903e2d434f15bf623bcf3b7d2202/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:52.886821 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da7fbf-5033-44a7-9ccf-d1d779b593e0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:52.887144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da7fbf-5033-44a7-9ccf-d1d779b593e0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:52.887373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57da7fbf-5033-44a7-9ccf-d1d779b593e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:52.922291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-57da7fbf-5033-44a7-9ccf-d1d779b593e0 None None] MFA Rules not processed for user `1020903e2d434f15bf623bcf3b7d2202`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:52.923560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-57da7fbf-5033-44a7-9ccf-d1d779b593e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:53.018935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1087/2182] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:53.031726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963c4c1-09b3-4a43-96d4-0212c4b36bf7 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:53.034397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e963c4c1-09b3-4a43-96d4-0212c4b36bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:53.097653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:53.128011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963c4c1-09b3-4a43-96d4-0212c4b36bf7 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:53.132732 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426b2aa9-686c-4f89-a3e5-c3bbff2147b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:53.133624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b2aa9-686c-4f89-a3e5-c3bbff2147b9 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:53.133952 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b2aa9-686c-4f89-a3e5-c3bbff2147b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:53.134271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426b2aa9-686c-4f89-a3e5-c3bbff2147b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:53.145322 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b2aa9-686c-4f89-a3e5-c3bbff2147b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:53.146899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426b2aa9-686c-4f89-a3e5-c3bbff2147b9 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:53.168290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1096/2183] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:53.179673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da7fbf-5033-44a7-9ccf-d1d779b593e0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:53.306884 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57da7fbf-5033-44a7-9ccf-d1d779b593e0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:53.321746 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf36c2c-2dc1-4c90-8604-561d05215e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:53.323516 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf36c2c-2dc1-4c90-8604-561d05215e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:53.323516 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf36c2c-2dc1-4c90-8604-561d05215e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:53.323516 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf36c2c-2dc1-4c90-8604-561d05215e21 admin admin] PATH_INFO: `/v3/users/1020903e2d434f15bf623bcf3b7d2202` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:53.324645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf36c2c-2dc1-4c90-8604-561d05215e21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1020903e2d434f15bf623bcf3b7d2202)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:53.327456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf36c2c-2dc1-4c90-8604-561d05215e21 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:53.382279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:53.453809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ddf36c2c-2dc1-4c90-8604-561d05215e21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1020903e2d434f15bf623bcf3b7d2202'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:53.454793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ddf36c2c-2dc1-4c90-8604-561d05215e21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1020903e2d434f15bf623bcf3b7d2202'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:53.479382 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ddf36c2c-2dc1-4c90-8604-561d05215e21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1020903e2d434f15bf623bcf3b7d2202'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:53.488766 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ddf36c2c-2dc1-4c90-8604-561d05215e21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1020903e2d434f15bf623bcf3b7d2202'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:53.489527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ddf36c2c-2dc1-4c90-8604-561d05215e21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1020903e2d434f15bf623bcf3b7d2202'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:53.504869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1088/2184] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:53 2022] DELETE /identity/v3/users/1020903e2d434f15bf623bcf3b7d2202 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:53.509890 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1324f5-c4e2-4cfe-ab96-63792ca88c07 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:53.510473 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1324f5-c4e2-4cfe-ab96-63792ca88c07 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:53.510571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1324f5-c4e2-4cfe-ab96-63792ca88c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:53.530674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-6c1324f5-c4e2-4cfe-ab96-63792ca88c07 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:53.532460 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6c1324f5-c4e2-4cfe-ab96-63792ca88c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:53.624827 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1097/2185] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:53.634463 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf36c2c-2dc1-4c90-8604-561d05215e21 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:53.635886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ddf36c2c-2dc1-4c90-8604-561d05215e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:53.703206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf36c2c-2dc1-4c90-8604-561d05215e21 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:53.707692 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2880f9-d293-4af4-b876-cac583086335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:53.708314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2880f9-d293-4af4-b876-cac583086335 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:53.708596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2880f9-d293-4af4-b876-cac583086335 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:53.708870 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2880f9-d293-4af4-b876-cac583086335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:53.719047 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2880f9-d293-4af4-b876-cac583086335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:53.720605 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2880f9-d293-4af4-b876-cac583086335 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:53.731085 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1089/2186] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:54.014103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1324f5-c4e2-4cfe-ab96-63792ca88c07 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:54.015586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6c1324f5-c4e2-4cfe-ab96-63792ca88c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:54.029052 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1324f5-c4e2-4cfe-ab96-63792ca88c07 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:54.033968 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6fa6b-c0b6-41ba-a564-3481b42ed4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:54.034635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6fa6b-c0b6-41ba-a564-3481b42ed4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:54.034912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6fa6b-c0b6-41ba-a564-3481b42ed4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:54.035181 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6fa6b-c0b6-41ba-a564-3481b42ed4ba admin admin] PATH_INFO: `/v3/projects/f08b45020e27401bb97912ac9e1f15bd` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:54.036448 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6fa6b-c0b6-41ba-a564-3481b42ed4ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=f08b45020e27401bb97912ac9e1f15bd)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:54.038376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6fa6b-c0b6-41ba-a564-3481b42ed4ba admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:54.092044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-12b6fa6b-c0b6-41ba-a564-3481b42ed4ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f08b45020e27401bb97912ac9e1f15bd', 'request_id': 'req-12b6fa6b-c0b6-41ba-a564-3481b42ed4ba'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:54.099809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-12b6fa6b-c0b6-41ba-a564-3481b42ed4ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f08b45020e27401bb97912ac9e1f15bd', 'request_id': 'req-12b6fa6b-c0b6-41ba-a564-3481b42ed4ba'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:54.107229 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-12b6fa6b-c0b6-41ba-a564-3481b42ed4ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f08b45020e27401bb97912ac9e1f15bd', 'request_id': 'req-12b6fa6b-c0b6-41ba-a564-3481b42ed4ba'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:54.119094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-12b6fa6b-c0b6-41ba-a564-3481b42ed4ba admin admin] The token cache is being invalidate because project f08b45020e27401bb97912ac9e1f15bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:54.119473 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-12b6fa6b-c0b6-41ba-a564-3481b42ed4ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:54.120307 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-12b6fa6b-c0b6-41ba-a564-3481b42ed4ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:54.122670 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1098/2187] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:54 2022] DELETE /identity/v3/projects/f08b45020e27401bb97912ac9e1f15bd => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:54.128418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9c68b-dc4f-4eb9-aad7-849ab4a6dfa5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:54.128712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9c68b-dc4f-4eb9-aad7-849ab4a6dfa5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:54.128988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9c68b-dc4f-4eb9-aad7-849ab4a6dfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:54.149447 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-25c9c68b-dc4f-4eb9-aad7-849ab4a6dfa5 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:54.150766 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-25c9c68b-dc4f-4eb9-aad7-849ab4a6dfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:54.215969 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1090/2188] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:54.223655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13389c91-874b-42f6-812f-5f70876f9d8f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:54.223940 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13389c91-874b-42f6-812f-5f70876f9d8f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:54.224218 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13389c91-874b-42f6-812f-5f70876f9d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:54.245212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-13389c91-874b-42f6-812f-5f70876f9d8f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:54.246478 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-13389c91-874b-42f6-812f-5f70876f9d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:54.255347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1099/2189] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:54.259675 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9c68b-dc4f-4eb9-aad7-849ab4a6dfa5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:54.269817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9c68b-dc4f-4eb9-aad7-849ab4a6dfa5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:54.275842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb4dac-1fbd-4c8f-a568-16845fa7c092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:54.276581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb4dac-1fbd-4c8f-a568-16845fa7c092 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:54.276938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb4dac-1fbd-4c8f-a568-16845fa7c092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:54.277222 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb4dac-1fbd-4c8f-a568-16845fa7c092 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:54.277890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb4dac-1fbd-4c8f-a568-16845fa7c092 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:54.278238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb4dac-1fbd-4c8f-a568-16845fa7c092 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:54.279957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb4dac-1fbd-4c8f-a568-16845fa7c092 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:54.296626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1091/2190] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:27:54.298552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13389c91-874b-42f6-812f-5f70876f9d8f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:54.308303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13389c91-874b-42f6-812f-5f70876f9d8f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:54.315899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef068c-5403-4dda-bdb6-6a863c895915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:54.316541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef068c-5403-4dda-bdb6-6a863c895915 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:54.316819 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef068c-5403-4dda-bdb6-6a863c895915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:54.317088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bef068c-5403-4dda-bdb6-6a863c895915 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:54.317729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef068c-5403-4dda-bdb6-6a863c895915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:54.320058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bef068c-5403-4dda-bdb6-6a863c895915 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:54.360289 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1100/2191] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:54 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:54.363586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb4dac-1fbd-4c8f-a568-16845fa7c092 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:54.379626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb4dac-1fbd-4c8f-a568-16845fa7c092 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:54.413015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:54.474251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c282e-ea6a-4daa-9caa-168cc74c2675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:54.475008 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c282e-ea6a-4daa-9caa-168cc74c2675 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:54.477464 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c282e-ea6a-4daa-9caa-168cc74c2675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:54.477464 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918c282e-ea6a-4daa-9caa-168cc74c2675 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:54.477464 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c282e-ea6a-4daa-9caa-168cc74c2675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:54.480000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918c282e-ea6a-4daa-9caa-168cc74c2675 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:54.532920 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1092/2192] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:54 2022] POST /identity/v3/users => generated 287 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:54.536973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef068c-5403-4dda-bdb6-6a863c895915 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:54.549857 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bef068c-5403-4dda-bdb6-6a863c895915 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:54.556521 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34fdca-45c9-4b60-937d-6489923f1de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:54.557181 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34fdca-45c9-4b60-937d-6489923f1de1 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:54.570459 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34fdca-45c9-4b60-937d-6489923f1de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:54.570755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34fdca-45c9-4b60-937d-6489923f1de1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:54.571461 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34fdca-45c9-4b60-937d-6489923f1de1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:54.573187 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34fdca-45c9-4b60-937d-6489923f1de1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:54.587867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1101/2193] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:54.592707 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c282e-ea6a-4daa-9caa-168cc74c2675 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:54.615154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918c282e-ea6a-4daa-9caa-168cc74c2675 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:54.621353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da31b1-da82-4b4c-a432-ac7319277fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:54.622213 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da31b1-da82-4b4c-a432-ac7319277fae admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:54.622527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da31b1-da82-4b4c-a432-ac7319277fae admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:54.622803 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da31b1-da82-4b4c-a432-ac7319277fae admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:54.623909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da31b1-da82-4b4c-a432-ac7319277fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:54.625906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da31b1-da82-4b4c-a432-ac7319277fae admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:54.656403 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1093/2194] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:54.658598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34fdca-45c9-4b60-937d-6489923f1de1 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:54.678177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34fdca-45c9-4b60-937d-6489923f1de1 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:54.689328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c49ee-a499-4488-8f74-33d5be7e5887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:54.689328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c49ee-a499-4488-8f74-33d5be7e5887 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:54.689937 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c49ee-a499-4488-8f74-33d5be7e5887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:54.689937 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c49ee-a499-4488-8f74-33d5be7e5887 admin admin] PATH_INFO: `/v3/projects/c347ff9a5e9c44dd8c8b9aaaff9fba07/users/b6f4139199014e9dab4ccd7764fd6131/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:54.722170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c49ee-a499-4488-8f74-33d5be7e5887 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c347ff9a5e9c44dd8c8b9aaaff9fba07, user_id=b6f4139199014e9dab4ccd7764fd6131, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:54.725063 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c49ee-a499-4488-8f74-33d5be7e5887 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:54.727656 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:54.745550 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1102/2195] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:54 2022] PUT /identity/v3/projects/c347ff9a5e9c44dd8c8b9aaaff9fba07/users/b6f4139199014e9dab4ccd7764fd6131/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:54.755693 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dba8e4-932a-4bfb-b8ca-bd49270df80e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:54.756011 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dba8e4-932a-4bfb-b8ca-bd49270df80e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:54.756243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3dba8e4-932a-4bfb-b8ca-bd49270df80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:54.794170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b3dba8e4-932a-4bfb-b8ca-bd49270df80e None None] MFA Rules not processed for user `b6f4139199014e9dab4ccd7764fd6131`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:54.796015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b3dba8e4-932a-4bfb-b8ca-bd49270df80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:54.868686 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1094/2196] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:54.882643 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c49ee-a499-4488-8f74-33d5be7e5887 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:54.884450 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f48c49ee-a499-4488-8f74-33d5be7e5887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:54.951892 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c49ee-a499-4488-8f74-33d5be7e5887 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:54.959135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1736594-fcc3-4fe8-b7fb-387c93ae1cdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:54.959765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1736594-fcc3-4fe8-b7fb-387c93ae1cdd service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:54.960053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1736594-fcc3-4fe8-b7fb-387c93ae1cdd service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:54.960333 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1736594-fcc3-4fe8-b7fb-387c93ae1cdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:54.971313 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1736594-fcc3-4fe8-b7fb-387c93ae1cdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:54.973037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1736594-fcc3-4fe8-b7fb-387c93ae1cdd service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:54.986821 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1103/2197] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:55.009278 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dba8e4-932a-4bfb-b8ca-bd49270df80e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:55.060344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1736594-fcc3-4fe8-b7fb-387c93ae1cdd service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:55.061902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a1736594-fcc3-4fe8-b7fb-387c93ae1cdd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:55.083934 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3dba8e4-932a-4bfb-b8ca-bd49270df80e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:55.089597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf1dfd9-8f6b-4902-b9d7-af05f5552709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:55.090248 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1dfd9-8f6b-4902-b9d7-af05f5552709 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:55.091050 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1dfd9-8f6b-4902-b9d7-af05f5552709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:55.091831 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf1dfd9-8f6b-4902-b9d7-af05f5552709 admin admin] PATH_INFO: `/v3/users/b6f4139199014e9dab4ccd7764fd6131` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:55.093346 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1dfd9-8f6b-4902-b9d7-af05f5552709 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6f4139199014e9dab4ccd7764fd6131)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:55.095778 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf1dfd9-8f6b-4902-b9d7-af05f5552709 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:55.133735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1736594-fcc3-4fe8-b7fb-387c93ae1cdd service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:55.140110 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69aad1f3-3843-4e4d-a03e-c69facf2ff0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:55.140496 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aad1f3-3843-4e4d-a03e-c69facf2ff0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:55.140824 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aad1f3-3843-4e4d-a03e-c69facf2ff0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:55.141293 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69aad1f3-3843-4e4d-a03e-c69facf2ff0a admin admin] PATH_INFO: `/v3/users/f21bd5d9b09847cd9a19ebc1c80da10a` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:55.142774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aad1f3-3843-4e4d-a03e-c69facf2ff0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f21bd5d9b09847cd9a19ebc1c80da10a)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:55.143333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:55.145173 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69aad1f3-3843-4e4d-a03e-c69facf2ff0a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:55.150288 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5bf1dfd9-8f6b-4902-b9d7-af05f5552709 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6f4139199014e9dab4ccd7764fd6131'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:55.151590 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5bf1dfd9-8f6b-4902-b9d7-af05f5552709 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6f4139199014e9dab4ccd7764fd6131'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:55.166294 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5bf1dfd9-8f6b-4902-b9d7-af05f5552709 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6f4139199014e9dab4ccd7764fd6131'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:55.179654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5bf1dfd9-8f6b-4902-b9d7-af05f5552709 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6f4139199014e9dab4ccd7764fd6131'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:55.181257 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5bf1dfd9-8f6b-4902-b9d7-af05f5552709 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6f4139199014e9dab4ccd7764fd6131'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:55.197423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1095/2198] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:55 2022] DELETE /identity/v3/users/b6f4139199014e9dab4ccd7764fd6131 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:55.202614 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc6b35-dcf2-44fd-8bf6-3dd70f4801d8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:55.202917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc6b35-dcf2-44fd-8bf6-3dd70f4801d8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:55.203191 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc6b35-dcf2-44fd-8bf6-3dd70f4801d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:55.208115 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-69aad1f3-3843-4e4d-a03e-c69facf2ff0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f21bd5d9b09847cd9a19ebc1c80da10a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:55.218135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-69aad1f3-3843-4e4d-a03e-c69facf2ff0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f21bd5d9b09847cd9a19ebc1c80da10a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:55.224424 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6abc6b35-dcf2-44fd-8bf6-3dd70f4801d8 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:55.226078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6abc6b35-dcf2-44fd-8bf6-3dd70f4801d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:55.236073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-69aad1f3-3843-4e4d-a03e-c69facf2ff0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f21bd5d9b09847cd9a19ebc1c80da10a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:55.236913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-69aad1f3-3843-4e4d-a03e-c69facf2ff0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f21bd5d9b09847cd9a19ebc1c80da10a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:55.248405 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-69aad1f3-3843-4e4d-a03e-c69facf2ff0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f21bd5d9b09847cd9a19ebc1c80da10a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:55.255964 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1104/2199] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:55 2022] DELETE /identity/v3/users/f21bd5d9b09847cd9a19ebc1c80da10a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:55.260243 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d090acc-3b56-4f0b-ade4-f1c44b8bb311 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:55.260528 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d090acc-3b56-4f0b-ade4-f1c44b8bb311 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:55.260800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d090acc-3b56-4f0b-ade4-f1c44b8bb311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:55.285071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7d090acc-3b56-4f0b-ade4-f1c44b8bb311 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:55.285071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7d090acc-3b56-4f0b-ade4-f1c44b8bb311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:55.295369 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:55.299853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1096/2200] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:55.310684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc6b35-dcf2-44fd-8bf6-3dd70f4801d8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:55.312129 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6abc6b35-dcf2-44fd-8bf6-3dd70f4801d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:55.375543 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1105/2201] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:55.387426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d090acc-3b56-4f0b-ade4-f1c44b8bb311 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:55.389704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7d090acc-3b56-4f0b-ade4-f1c44b8bb311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:55.392160 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc6b35-dcf2-44fd-8bf6-3dd70f4801d8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:55.462927 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d090acc-3b56-4f0b-ade4-f1c44b8bb311 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:55.476831 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bc6ea-d838-4bde-81f8-132cb93e8670 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:55.477621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bc6ea-d838-4bde-81f8-132cb93e8670 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:55.478014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bc6ea-d838-4bde-81f8-132cb93e8670 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:55.478415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4bc6ea-d838-4bde-81f8-132cb93e8670 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:55.488222 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9954202-51ff-497a-8b28-9a0fc7ff74df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:55.488868 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9954202-51ff-497a-8b28-9a0fc7ff74df service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:55.489145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9954202-51ff-497a-8b28-9a0fc7ff74df service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:55.489416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9954202-51ff-497a-8b28-9a0fc7ff74df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:55.491599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bc6ea-d838-4bde-81f8-132cb93e8670 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:55.493293 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4bc6ea-d838-4bde-81f8-132cb93e8670 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:55.505787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1106/2202] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:55.513641 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9954202-51ff-497a-8b28-9a0fc7ff74df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:55.515355 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9954202-51ff-497a-8b28-9a0fc7ff74df service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:55.525837 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1097/2203] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:55.923431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bc6ea-d838-4bde-81f8-132cb93e8670 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:55.926939 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4b4bc6ea-d838-4bde-81f8-132cb93e8670 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:55.943605 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4bc6ea-d838-4bde-81f8-132cb93e8670 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:55.949143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7ec94-fff2-4264-8a63-995028b288c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:55.949742 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7ec94-fff2-4264-8a63-995028b288c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:55.950029 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7ec94-fff2-4264-8a63-995028b288c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:55.950367 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d7ec94-fff2-4264-8a63-995028b288c0 admin admin] PATH_INFO: `/v3/projects/c6f2a1c9f08443c3a527c618c2a895fb` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:55.951618 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7ec94-fff2-4264-8a63-995028b288c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6f2a1c9f08443c3a527c618c2a895fb)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:55.954283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d7ec94-fff2-4264-8a63-995028b288c0 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:55.982182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9954202-51ff-497a-8b28-9a0fc7ff74df service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:55.984170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e9954202-51ff-497a-8b28-9a0fc7ff74df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:55.998939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9954202-51ff-497a-8b28-9a0fc7ff74df service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:56.007183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a3d7ec94-fff2-4264-8a63-995028b288c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6f2a1c9f08443c3a527c618c2a895fb', 'request_id': 'req-a3d7ec94-fff2-4264-8a63-995028b288c0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.015384 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:56.016427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a3d7ec94-fff2-4264-8a63-995028b288c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6f2a1c9f08443c3a527c618c2a895fb', 'request_id': 'req-a3d7ec94-fff2-4264-8a63-995028b288c0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.026639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a3d7ec94-fff2-4264-8a63-995028b288c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6f2a1c9f08443c3a527c618c2a895fb', 'request_id': 'req-a3d7ec94-fff2-4264-8a63-995028b288c0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.039086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a3d7ec94-fff2-4264-8a63-995028b288c0 admin admin] The token cache is being invalidate because project c6f2a1c9f08443c3a527c618c2a895fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:56.039376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a3d7ec94-fff2-4264-8a63-995028b288c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.040022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a3d7ec94-fff2-4264-8a63-995028b288c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.041519 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1107/2204] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:55 2022] DELETE /identity/v3/projects/c6f2a1c9f08443c3a527c618c2a895fb => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:56.045377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7ec94-fff2-4264-8a63-995028b288c0 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:56.046782 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a3d7ec94-fff2-4264-8a63-995028b288c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:56.070445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66245fdd-1701-42cf-9d91-de3dca852d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:56.071147 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66245fdd-1701-42cf-9d91-de3dca852d49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.071428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66245fdd-1701-42cf-9d91-de3dca852d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.071695 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66245fdd-1701-42cf-9d91-de3dca852d49 admin admin] PATH_INFO: `/v3/projects/c347ff9a5e9c44dd8c8b9aaaff9fba07` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.072807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66245fdd-1701-42cf-9d91-de3dca852d49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c347ff9a5e9c44dd8c8b9aaaff9fba07)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:56.074754 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66245fdd-1701-42cf-9d91-de3dca852d49 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:56.094813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:56.118716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d7ec94-fff2-4264-8a63-995028b288c0 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:56.135748 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-66245fdd-1701-42cf-9d91-de3dca852d49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c347ff9a5e9c44dd8c8b9aaaff9fba07', 'request_id': 'req-66245fdd-1701-42cf-9d91-de3dca852d49'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.147408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-66245fdd-1701-42cf-9d91-de3dca852d49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c347ff9a5e9c44dd8c8b9aaaff9fba07', 'request_id': 'req-66245fdd-1701-42cf-9d91-de3dca852d49'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.158803 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-66245fdd-1701-42cf-9d91-de3dca852d49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c347ff9a5e9c44dd8c8b9aaaff9fba07', 'request_id': 'req-66245fdd-1701-42cf-9d91-de3dca852d49'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.171133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-66245fdd-1701-42cf-9d91-de3dca852d49 admin admin] The token cache is being invalidate because project c347ff9a5e9c44dd8c8b9aaaff9fba07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:56.171418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-66245fdd-1701-42cf-9d91-de3dca852d49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.172068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-66245fdd-1701-42cf-9d91-de3dca852d49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.173962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1098/2205] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:55 2022] DELETE /identity/v3/projects/c347ff9a5e9c44dd8c8b9aaaff9fba07 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:56.180817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ca379-4061-42d0-aa43-08e10ad39d3e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.181484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ca379-4061-42d0-aa43-08e10ad39d3e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.182583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13ca379-4061-42d0-aa43-08e10ad39d3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.184839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a378e-a81f-45d5-9f96-b3616603a282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:56.185467 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a378e-a81f-45d5-9f96-b3616603a282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.185745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a378e-a81f-45d5-9f96-b3616603a282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.186036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0a378e-a81f-45d5-9f96-b3616603a282 admin admin] PATH_INFO: `/v3/users/8fb6c86a7b0340d0b48afef18d00e221` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.187316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a378e-a81f-45d5-9f96-b3616603a282 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fb6c86a7b0340d0b48afef18d00e221)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:56.189266 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0a378e-a81f-45d5-9f96-b3616603a282 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:56.209602 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d13ca379-4061-42d0-aa43-08e10ad39d3e None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:56.210979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d13ca379-4061-42d0-aa43-08e10ad39d3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:56.220459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1099/2206] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:56.229679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfbcb1-6b00-41f1-ac23-ab514605dd4f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.230327 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfbcb1-6b00-41f1-ac23-ab514605dd4f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.230327 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfbcb1-6b00-41f1-ac23-ab514605dd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.242046 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8a0a378e-a81f-45d5-9f96-b3616603a282 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fb6c86a7b0340d0b48afef18d00e221'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.250253 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8a0a378e-a81f-45d5-9f96-b3616603a282 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fb6c86a7b0340d0b48afef18d00e221'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.252991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-85bfbcb1-6b00-41f1-ac23-ab514605dd4f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:56.254506 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-85bfbcb1-6b00-41f1-ac23-ab514605dd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:56.262909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1100/2207] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:56.267071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8a0a378e-a81f-45d5-9f96-b3616603a282 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fb6c86a7b0340d0b48afef18d00e221'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.267245 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfbcb1-6b00-41f1-ac23-ab514605dd4f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:56.267993 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8a0a378e-a81f-45d5-9f96-b3616603a282 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fb6c86a7b0340d0b48afef18d00e221'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.276325 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8a0a378e-a81f-45d5-9f96-b3616603a282 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fb6c86a7b0340d0b48afef18d00e221'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.279219 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfbcb1-6b00-41f1-ac23-ab514605dd4f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:56.284856 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c67b7-613e-4674-bcef-ff09342a79ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:56.285199 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1108/2208] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:56 2022] DELETE /identity/v3/users/8fb6c86a7b0340d0b48afef18d00e221 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:56.286531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c67b7-613e-4674-bcef-ff09342a79ba admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.286894 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c67b7-613e-4674-bcef-ff09342a79ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.287272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c67b7-613e-4674-bcef-ff09342a79ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.288027 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c67b7-613e-4674-bcef-ff09342a79ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:56.288333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c67b7-613e-4674-bcef-ff09342a79ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:56.289990 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c67b7-613e-4674-bcef-ff09342a79ba admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:56.310088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1101/2209] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:27:56.311317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a378e-a81f-45d5-9f96-b3616603a282 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:56.312789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8a0a378e-a81f-45d5-9f96-b3616603a282 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:56.388391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0a378e-a81f-45d5-9f96-b3616603a282 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:56.393527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87137a07-05db-47af-a8d0-16300019934f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:56.394134 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87137a07-05db-47af-a8d0-16300019934f admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.394441 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87137a07-05db-47af-a8d0-16300019934f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.394710 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87137a07-05db-47af-a8d0-16300019934f admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.395487 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87137a07-05db-47af-a8d0-16300019934f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:56.397259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87137a07-05db-47af-a8d0-16300019934f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:56.413153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:56.432821 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1109/2210] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:56 2022] POST /identity/v3/projects => generated 375 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:56.433884 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c67b7-613e-4674-bcef-ff09342a79ba admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:56.445935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c67b7-613e-4674-bcef-ff09342a79ba admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:56.522708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500aea4-c325-457f-a341-089e24263a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:56.522708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500aea4-c325-457f-a341-089e24263a1b admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.522708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500aea4-c325-457f-a341-089e24263a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.522708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500aea4-c325-457f-a341-089e24263a1b admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.523224 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e500aea4-c325-457f-a341-089e24263a1b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:56.523224 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e500aea4-c325-457f-a341-089e24263a1b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:56.555413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1102/2211] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:56 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:56.556698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87137a07-05db-47af-a8d0-16300019934f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:56.568168 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87137a07-05db-47af-a8d0-16300019934f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:56.572800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0580a-8882-495b-be7b-f5980df56445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:56.573414 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0580a-8882-495b-be7b-f5980df56445 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.574081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0580a-8882-495b-be7b-f5980df56445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.574407 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf0580a-8882-495b-be7b-f5980df56445 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.575922 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf0580a-8882-495b-be7b-f5980df56445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:56.577775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf0580a-8882-495b-be7b-f5980df56445 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:56.594482 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1110/2212] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:56.598666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500aea4-c325-457f-a341-089e24263a1b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:56.632861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500aea4-c325-457f-a341-089e24263a1b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:56.642790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ea80c-9312-48e0-963b-1e1073883c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:56.643517 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ea80c-9312-48e0-963b-1e1073883c31 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.643932 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ea80c-9312-48e0-963b-1e1073883c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.643932 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29ea80c-9312-48e0-963b-1e1073883c31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.644589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ea80c-9312-48e0-963b-1e1073883c31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:56.646209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29ea80c-9312-48e0-963b-1e1073883c31 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:56.656914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0580a-8882-495b-be7b-f5980df56445 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:56.659022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2bf0580a-8882-495b-be7b-f5980df56445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:56.664765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:56.673742 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf0580a-8882-495b-be7b-f5980df56445 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:56.678107 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1103/2213] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:56 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:56.680671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ea80c-9312-48e0-963b-1e1073883c31 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:56.685559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825b443-3d75-453b-9182-08a18eb3d462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:56.691847 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825b443-3d75-453b-9182-08a18eb3d462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.692138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825b443-3d75-453b-9182-08a18eb3d462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.692667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825b443-3d75-453b-9182-08a18eb3d462 admin admin] PATH_INFO: `/v3/projects/fd735a58080d46d480bc84e8e24fd051` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.697088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825b443-3d75-453b-9182-08a18eb3d462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd735a58080d46d480bc84e8e24fd051)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:56.697088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825b443-3d75-453b-9182-08a18eb3d462 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:56.697291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29ea80c-9312-48e0-963b-1e1073883c31 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:56.703101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e1cf1-67c5-4949-a986-89c739e54560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:56.703780 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e1cf1-67c5-4949-a986-89c739e54560 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.704058 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e1cf1-67c5-4949-a986-89c739e54560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.705011 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e1cf1-67c5-4949-a986-89c739e54560 admin admin] PATH_INFO: `/v3/projects/40cadfb2bb8e4f5cbf604d9ce2887d90/users/ac3c334ba8794f579c13a177520e29aa/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.723097 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e1cf1-67c5-4949-a986-89c739e54560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40cadfb2bb8e4f5cbf604d9ce2887d90, user_id=ac3c334ba8794f579c13a177520e29aa, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:56.725982 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e1cf1-67c5-4949-a986-89c739e54560 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:56.744760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1104/2214] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:56 2022] PUT /identity/v3/projects/40cadfb2bb8e4f5cbf604d9ce2887d90/users/ac3c334ba8794f579c13a177520e29aa/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:56.757416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb31118-712b-45d3-8cc1-bb1f0c2cd3eb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.758388 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb31118-712b-45d3-8cc1-bb1f0c2cd3eb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.758388 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb31118-712b-45d3-8cc1-bb1f0c2cd3eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.775808 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f825b443-3d75-453b-9182-08a18eb3d462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd735a58080d46d480bc84e8e24fd051', 'request_id': 'req-f825b443-3d75-453b-9182-08a18eb3d462'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.787744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f825b443-3d75-453b-9182-08a18eb3d462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd735a58080d46d480bc84e8e24fd051', 'request_id': 'req-f825b443-3d75-453b-9182-08a18eb3d462'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.797471 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f825b443-3d75-453b-9182-08a18eb3d462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd735a58080d46d480bc84e8e24fd051', 'request_id': 'req-f825b443-3d75-453b-9182-08a18eb3d462'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.813189 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0eb31118-712b-45d3-8cc1-bb1f0c2cd3eb None None] MFA Rules not processed for user `ac3c334ba8794f579c13a177520e29aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:56.814468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0eb31118-712b-45d3-8cc1-bb1f0c2cd3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:56.827965 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f825b443-3d75-453b-9182-08a18eb3d462 admin admin] The token cache is being invalidate because project fd735a58080d46d480bc84e8e24fd051 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:56.828514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f825b443-3d75-453b-9182-08a18eb3d462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.829765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f825b443-3d75-453b-9182-08a18eb3d462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:56.832100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1111/2215] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:56 2022] DELETE /identity/v3/projects/fd735a58080d46d480bc84e8e24fd051 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:56.840274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d7422-8fe0-4423-8ae7-c8d88d1aae82 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.840476 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d7422-8fe0-4423-8ae7-c8d88d1aae82 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.841741 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d7422-8fe0-4423-8ae7-c8d88d1aae82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:56.882551 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a13d7422-8fe0-4423-8ae7-c8d88d1aae82 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:56.882551 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a13d7422-8fe0-4423-8ae7-c8d88d1aae82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:56.915529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1105/2216] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:56.927673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb31118-712b-45d3-8cc1-bb1f0c2cd3eb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:56.929470 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0eb31118-712b-45d3-8cc1-bb1f0c2cd3eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:56.962681 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1112/2217] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:56.971864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3dc82b-7df7-4202-b6fd-658312b10103 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:56.972148 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3dc82b-7df7-4202-b6fd-658312b10103 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:56.972420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3dc82b-7df7-4202-b6fd-658312b10103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:57.001291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7c3dc82b-7df7-4202-b6fd-658312b10103 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:57.002584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7c3dc82b-7df7-4202-b6fd-658312b10103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:57.009110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb31118-712b-45d3-8cc1-bb1f0c2cd3eb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:57.011943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1113/2218] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:57.017758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28087330-8357-4d7a-903a-860a84e3454d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:57.017758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28087330-8357-4d7a-903a-860a84e3454d service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:57.017758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28087330-8357-4d7a-903a-860a84e3454d service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:57.017758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28087330-8357-4d7a-903a-860a84e3454d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:57.018274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3dc82b-7df7-4202-b6fd-658312b10103 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:57.026332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28087330-8357-4d7a-903a-860a84e3454d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:57.027205 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3dc82b-7df7-4202-b6fd-658312b10103 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:57.027766 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28087330-8357-4d7a-903a-860a84e3454d service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:57.032561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e12c81-7043-47c6-86e9-5ae7b205a45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:57.033175 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e12c81-7043-47c6-86e9-5ae7b205a45b admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:57.033582 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e12c81-7043-47c6-86e9-5ae7b205a45b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:57.034005 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e12c81-7043-47c6-86e9-5ae7b205a45b admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:57.034853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e12c81-7043-47c6-86e9-5ae7b205a45b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:27:57.035223 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e12c81-7043-47c6-86e9-5ae7b205a45b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:57.036902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e12c81-7043-47c6-86e9-5ae7b205a45b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:57.037424 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1106/2219] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:57.050087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1114/2220] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:27:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:27:57.054734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28087330-8357-4d7a-903a-860a84e3454d service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:57.068049 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28087330-8357-4d7a-903a-860a84e3454d service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:57.073905 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debda9f1-36ea-44e5-89fc-9b8cdacb3814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:57.074538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debda9f1-36ea-44e5-89fc-9b8cdacb3814 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:57.074814 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debda9f1-36ea-44e5-89fc-9b8cdacb3814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:57.075078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debda9f1-36ea-44e5-89fc-9b8cdacb3814 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:57.076272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debda9f1-36ea-44e5-89fc-9b8cdacb3814 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:57.078185 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debda9f1-36ea-44e5-89fc-9b8cdacb3814 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:57.113219 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1107/2221] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:57 2022] POST /identity/v3/projects => generated 387 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:27:57.115734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e12c81-7043-47c6-86e9-5ae7b205a45b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:57.127908 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e12c81-7043-47c6-86e9-5ae7b205a45b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:57.182932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:57.202639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e415da-0f7f-4075-8f72-b3fcfc1e7350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:57.203169 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e415da-0f7f-4075-8f72-b3fcfc1e7350 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:57.203450 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e415da-0f7f-4075-8f72-b3fcfc1e7350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:57.203722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e415da-0f7f-4075-8f72-b3fcfc1e7350 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:57.204467 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e415da-0f7f-4075-8f72-b3fcfc1e7350 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:57.206422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e415da-0f7f-4075-8f72-b3fcfc1e7350 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:57.236030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1115/2222] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:57 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:57.237477 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debda9f1-36ea-44e5-89fc-9b8cdacb3814 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:57.251566 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debda9f1-36ea-44e5-89fc-9b8cdacb3814 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:57.258619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13672227-ca11-4047-a925-7088b18fd084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:57.258619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13672227-ca11-4047-a925-7088b18fd084 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:57.258619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13672227-ca11-4047-a925-7088b18fd084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:57.258619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13672227-ca11-4047-a925-7088b18fd084 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:57.259697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13672227-ca11-4047-a925-7088b18fd084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:57.261614 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13672227-ca11-4047-a925-7088b18fd084 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:57.277426 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1108/2223] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:27:57.279430 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e415da-0f7f-4075-8f72-b3fcfc1e7350 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:57.290469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e415da-0f7f-4075-8f72-b3fcfc1e7350 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:57.297679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2117771-ddcc-4c8d-bf73-ed8a69005d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:57.298417 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2117771-ddcc-4c8d-bf73-ed8a69005d5b admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:57.298731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2117771-ddcc-4c8d-bf73-ed8a69005d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:57.299028 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2117771-ddcc-4c8d-bf73-ed8a69005d5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:57.299675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2117771-ddcc-4c8d-bf73-ed8a69005d5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:57.301277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2117771-ddcc-4c8d-bf73-ed8a69005d5b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:57.315783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1116/2224] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:57.320098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13672227-ca11-4047-a925-7088b18fd084 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:57.330712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13672227-ca11-4047-a925-7088b18fd084 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:57.344457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81a08a-8feb-49d8-9737-1f9b7d3127db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:57.345263 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81a08a-8feb-49d8-9737-1f9b7d3127db admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:57.345671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81a08a-8feb-49d8-9737-1f9b7d3127db admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:57.346037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b81a08a-8feb-49d8-9737-1f9b7d3127db admin admin] PATH_INFO: `/v3/projects/2fb01bf19c8d40c6b6f5eebdac53f9ba/users/2bbcb3cea9004dbc9fe19a9533ead2d9/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:57.366912 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81a08a-8feb-49d8-9737-1f9b7d3127db admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fb01bf19c8d40c6b6f5eebdac53f9ba, user_id=2bbcb3cea9004dbc9fe19a9533ead2d9, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:57.373722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b81a08a-8feb-49d8-9737-1f9b7d3127db admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:57.393838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1109/2225] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:57 2022] PUT /identity/v3/projects/2fb01bf19c8d40c6b6f5eebdac53f9ba/users/2bbcb3cea9004dbc9fe19a9533ead2d9/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:57.406448 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87bfbe-741a-4a95-a3f8-81987167af01 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:57.406448 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87bfbe-741a-4a95-a3f8-81987167af01 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:57.406448 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87bfbe-741a-4a95-a3f8-81987167af01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:57.440575 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4b87bfbe-741a-4a95-a3f8-81987167af01 None None] MFA Rules not processed for user `2bbcb3cea9004dbc9fe19a9533ead2d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:57.441805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4b87bfbe-741a-4a95-a3f8-81987167af01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:57.524510 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1117/2226] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:57.529339 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81a08a-8feb-49d8-9737-1f9b7d3127db admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:57.543458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:57.597136 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b81a08a-8feb-49d8-9737-1f9b7d3127db admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:57.602200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4beb521-77f6-4d61-a601-884e1ec862d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:57.602848 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4beb521-77f6-4d61-a601-884e1ec862d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:57.603127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4beb521-77f6-4d61-a601-884e1ec862d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:57.603397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4beb521-77f6-4d61-a601-884e1ec862d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:57.604046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4beb521-77f6-4d61-a601-884e1ec862d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:57.605966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4beb521-77f6-4d61-a601-884e1ec862d8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:57.634465 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1110/2227] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:27:57 2022] POST /identity/v3/projects => generated 387 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:57.639707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87bfbe-741a-4a95-a3f8-81987167af01 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:57.650974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87bfbe-741a-4a95-a3f8-81987167af01 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:57.720032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ced9d7-d690-4cd2-8b56-1b948e6fe7cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:57.720648 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ced9d7-d690-4cd2-8b56-1b948e6fe7cb admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:57.720929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ced9d7-d690-4cd2-8b56-1b948e6fe7cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:57.721204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ced9d7-d690-4cd2-8b56-1b948e6fe7cb admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:57.721836 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ced9d7-d690-4cd2-8b56-1b948e6fe7cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:57.723834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ced9d7-d690-4cd2-8b56-1b948e6fe7cb admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:57.748335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1118/2228] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:27:57 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:27:57.756082 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4beb521-77f6-4d61-a601-884e1ec862d8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:57.775229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4beb521-77f6-4d61-a601-884e1ec862d8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:57.783348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6fd74-ffe1-4bfb-807a-4d481db1c4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:57.783774 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6fd74-ffe1-4bfb-807a-4d481db1c4bb admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:57.784050 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6fd74-ffe1-4bfb-807a-4d481db1c4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:57.784318 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b6fd74-ffe1-4bfb-807a-4d481db1c4bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:57.784961 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b6fd74-ffe1-4bfb-807a-4d481db1c4bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:57.786582 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b6fd74-ffe1-4bfb-807a-4d481db1c4bb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:57.802059 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1111/2229] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:57.803744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ced9d7-d690-4cd2-8b56-1b948e6fe7cb admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:57.818454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ced9d7-d690-4cd2-8b56-1b948e6fe7cb admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:57.823739 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ead9a-0090-4166-8deb-970c89be2c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:57.824401 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ead9a-0090-4166-8deb-970c89be2c47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:57.824675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ead9a-0090-4166-8deb-970c89be2c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:57.824945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715ead9a-0090-4166-8deb-970c89be2c47 admin admin] PATH_INFO: `/v3/projects/fe2e407539e74522906ec4d530a6dbc0/users/7a28a5b88d214a9f94bd43d8d1a9207d/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:57.845404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ead9a-0090-4166-8deb-970c89be2c47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2e407539e74522906ec4d530a6dbc0, user_id=7a28a5b88d214a9f94bd43d8d1a9207d, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:57.848083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715ead9a-0090-4166-8deb-970c89be2c47 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:57.871055 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1119/2230] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:57 2022] PUT /identity/v3/projects/fe2e407539e74522906ec4d530a6dbc0/users/7a28a5b88d214a9f94bd43d8d1a9207d/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:57.875507 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6fd74-ffe1-4bfb-807a-4d481db1c4bb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:57.892238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b6fd74-ffe1-4bfb-807a-4d481db1c4bb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:57.985877 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040b89d1-0c63-4be6-b40c-79474b74b8ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:57.986738 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b89d1-0c63-4be6-b40c-79474b74b8ad admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:57.987019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b89d1-0c63-4be6-b40c-79474b74b8ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:57.987293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040b89d1-0c63-4be6-b40c-79474b74b8ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:57.987944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b89d1-0c63-4be6-b40c-79474b74b8ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:57.989627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040b89d1-0c63-4be6-b40c-79474b74b8ad admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:58.000463 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1112/2231] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:27:57 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:27:58.005152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ead9a-0090-4166-8deb-970c89be2c47 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:58.022802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715ead9a-0090-4166-8deb-970c89be2c47 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:58.027807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b2c2b-3fb5-429f-b7af-08aea5714d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:58.028482 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b2c2b-3fb5-429f-b7af-08aea5714d99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:58.029278 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b2c2b-3fb5-429f-b7af-08aea5714d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:58.029552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2b2c2b-3fb5-429f-b7af-08aea5714d99 admin admin] PATH_INFO: `/v3/projects/fe2e407539e74522906ec4d530a6dbc0/users/7a28a5b88d214a9f94bd43d8d1a9207d/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:58.031540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b2c2b-3fb5-429f-b7af-08aea5714d99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2e407539e74522906ec4d530a6dbc0, user_id=7a28a5b88d214a9f94bd43d8d1a9207d, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:58.034917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2b2c2b-3fb5-429f-b7af-08aea5714d99 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:58.046925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1120/2232] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:27:58 2022] PUT /identity/v3/projects/fe2e407539e74522906ec4d530a6dbc0/users/7a28a5b88d214a9f94bd43d8d1a9207d/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:58.058929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b14fe-3138-4c00-8138-b8ed8d1efef9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:58.059283 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b14fe-3138-4c00-8138-b8ed8d1efef9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:58.059700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b14fe-3138-4c00-8138-b8ed8d1efef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:58.102910 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-cf6b14fe-3138-4c00-8138-b8ed8d1efef9 None None] MFA Rules not processed for user `7a28a5b88d214a9f94bd43d8d1a9207d`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:58.105271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cf6b14fe-3138-4c00-8138-b8ed8d1efef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:58.229344 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1113/2233] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:58 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:27:58.243102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b2c2b-3fb5-429f-b7af-08aea5714d99 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:58.255395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:58.305876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2b2c2b-3fb5-429f-b7af-08aea5714d99 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:58.309442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70019d41-8a37-4f5a-815a-de17fadbc7c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:58.310609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70019d41-8a37-4f5a-815a-de17fadbc7c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:58.310884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70019d41-8a37-4f5a-815a-de17fadbc7c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:58.311151 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70019d41-8a37-4f5a-815a-de17fadbc7c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:58.320477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70019d41-8a37-4f5a-815a-de17fadbc7c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:58.321921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70019d41-8a37-4f5a-815a-de17fadbc7c1 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:58.332886 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1121/2234] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:58 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:27:58.401348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b14fe-3138-4c00-8138-b8ed8d1efef9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:58.411370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:58.478833 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b14fe-3138-4c00-8138-b8ed8d1efef9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:58.483835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7456f8a-e992-4c2a-976f-815f0d60837c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:58.484551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7456f8a-e992-4c2a-976f-815f0d60837c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:58.484551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7456f8a-e992-4c2a-976f-815f0d60837c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:58.485113 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7456f8a-e992-4c2a-976f-815f0d60837c admin admin] PATH_INFO: `/v3/users/2bbcb3cea9004dbc9fe19a9533ead2d9` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:58.486161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7456f8a-e992-4c2a-976f-815f0d60837c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2bbcb3cea9004dbc9fe19a9533ead2d9)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:58.488099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7456f8a-e992-4c2a-976f-815f0d60837c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:58.553939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f7456f8a-e992-4c2a-976f-815f0d60837c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2bbcb3cea9004dbc9fe19a9533ead2d9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:58.554900 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f7456f8a-e992-4c2a-976f-815f0d60837c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2bbcb3cea9004dbc9fe19a9533ead2d9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:58.578760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f7456f8a-e992-4c2a-976f-815f0d60837c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2bbcb3cea9004dbc9fe19a9533ead2d9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:58.586615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f7456f8a-e992-4c2a-976f-815f0d60837c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2bbcb3cea9004dbc9fe19a9533ead2d9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:58.587273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f7456f8a-e992-4c2a-976f-815f0d60837c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2bbcb3cea9004dbc9fe19a9533ead2d9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:58.601624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1114/2235] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:58 2022] DELETE /identity/v3/users/2bbcb3cea9004dbc9fe19a9533ead2d9 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:27:58.607203 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1d977-09e0-4392-95b9-a4113ae78ef3 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:58.607712 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1d977-09e0-4392-95b9-a4113ae78ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:58.608020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b1d977-09e0-4392-95b9-a4113ae78ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:58.628186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-21b1d977-09e0-4392-95b9-a4113ae78ef3 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:27:58.629431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-21b1d977-09e0-4392-95b9-a4113ae78ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:58.701502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1122/2236] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:27:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:27:58.711503 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7456f8a-e992-4c2a-976f-815f0d60837c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:58.712959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f7456f8a-e992-4c2a-976f-815f0d60837c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:58.775911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7456f8a-e992-4c2a-976f-815f0d60837c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:58.786790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72707743-5597-4785-b010-a50e8c9841f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:58.787426 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72707743-5597-4785-b010-a50e8c9841f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:58.787705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72707743-5597-4785-b010-a50e8c9841f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:58.787988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72707743-5597-4785-b010-a50e8c9841f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:58.799054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72707743-5597-4785-b010-a50e8c9841f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:58.800657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72707743-5597-4785-b010-a50e8c9841f8 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:58.811456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1115/2237] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:27:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:27:59.300374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1d977-09e0-4392-95b9-a4113ae78ef3 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:59.301835 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-21b1d977-09e0-4392-95b9-a4113ae78ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:27:59.315769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b1d977-09e0-4392-95b9-a4113ae78ef3 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:59.321363 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba23d2-2126-4a64-b691-cdfd463156ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:59.322025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba23d2-2126-4a64-b691-cdfd463156ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:59.322343 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba23d2-2126-4a64-b691-cdfd463156ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:59.322625 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcba23d2-2126-4a64-b691-cdfd463156ad admin admin] PATH_INFO: `/v3/projects/2fb01bf19c8d40c6b6f5eebdac53f9ba` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:59.323910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba23d2-2126-4a64-b691-cdfd463156ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fb01bf19c8d40c6b6f5eebdac53f9ba)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:59.326494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcba23d2-2126-4a64-b691-cdfd463156ad admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:59.416291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dcba23d2-2126-4a64-b691-cdfd463156ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fb01bf19c8d40c6b6f5eebdac53f9ba', 'request_id': 'req-dcba23d2-2126-4a64-b691-cdfd463156ad'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:59.423850 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dcba23d2-2126-4a64-b691-cdfd463156ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fb01bf19c8d40c6b6f5eebdac53f9ba', 'request_id': 'req-dcba23d2-2126-4a64-b691-cdfd463156ad'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:59.436071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dcba23d2-2126-4a64-b691-cdfd463156ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fb01bf19c8d40c6b6f5eebdac53f9ba', 'request_id': 'req-dcba23d2-2126-4a64-b691-cdfd463156ad'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:59.454366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dcba23d2-2126-4a64-b691-cdfd463156ad admin admin] The token cache is being invalidate because project 2fb01bf19c8d40c6b6f5eebdac53f9ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:27:59.454683 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dcba23d2-2126-4a64-b691-cdfd463156ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:59.455445 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dcba23d2-2126-4a64-b691-cdfd463156ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:59.456987 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1123/2238] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:27:59 2022] DELETE /identity/v3/projects/2fb01bf19c8d40c6b6f5eebdac53f9ba => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:27:59.461084 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72707743-5597-4785-b010-a50e8c9841f8 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:27:59.475871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72707743-5597-4785-b010-a50e8c9841f8 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:27:59.562420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:27:59.636231 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afd664-0d14-45ff-b770-6376889556b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:27:59.637044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afd664-0d14-45ff-b770-6376889556b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:27:59.637321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afd664-0d14-45ff-b770-6376889556b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:27:59.637591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afd664-0d14-45ff-b770-6376889556b2 admin admin] PATH_INFO: `/v3/users/7a28a5b88d214a9f94bd43d8d1a9207d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:27:59.640120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3afd664-0d14-45ff-b770-6376889556b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a28a5b88d214a9f94bd43d8d1a9207d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:27:59.642701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3afd664-0d14-45ff-b770-6376889556b2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:27:59.705740 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c3afd664-0d14-45ff-b770-6376889556b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a28a5b88d214a9f94bd43d8d1a9207d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:59.706616 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c3afd664-0d14-45ff-b770-6376889556b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a28a5b88d214a9f94bd43d8d1a9207d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:59.724068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c3afd664-0d14-45ff-b770-6376889556b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a28a5b88d214a9f94bd43d8d1a9207d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:59.734604 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c3afd664-0d14-45ff-b770-6376889556b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a28a5b88d214a9f94bd43d8d1a9207d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:59.735234 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c3afd664-0d14-45ff-b770-6376889556b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a28a5b88d214a9f94bd43d8d1a9207d'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:27:59.755191 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1116/2239] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:27:59 2022] DELETE /identity/v3/users/7a28a5b88d214a9f94bd43d8d1a9207d => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:00.286421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba23d2-2126-4a64-b691-cdfd463156ad admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:00.288722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-dcba23d2-2126-4a64-b691-cdfd463156ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:00.311785 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcba23d2-2126-4a64-b691-cdfd463156ad admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:00.370582 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:00.427461 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ecf0c5-a959-4ca5-9d81-9c258afe255d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:00.428454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecf0c5-a959-4ca5-9d81-9c258afe255d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:00.429423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecf0c5-a959-4ca5-9d81-9c258afe255d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:00.429893 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ecf0c5-a959-4ca5-9d81-9c258afe255d admin admin] PATH_INFO: `/v3/projects/fe2e407539e74522906ec4d530a6dbc0` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:00.431638 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ecf0c5-a959-4ca5-9d81-9c258afe255d admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe2e407539e74522906ec4d530a6dbc0)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:00.434778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ecf0c5-a959-4ca5-9d81-9c258afe255d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:00.535300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-18ecf0c5-a959-4ca5-9d81-9c258afe255d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe2e407539e74522906ec4d530a6dbc0', 'request_id': 'req-18ecf0c5-a959-4ca5-9d81-9c258afe255d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:00.543215 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-18ecf0c5-a959-4ca5-9d81-9c258afe255d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe2e407539e74522906ec4d530a6dbc0', 'request_id': 'req-18ecf0c5-a959-4ca5-9d81-9c258afe255d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:00.551922 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-18ecf0c5-a959-4ca5-9d81-9c258afe255d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe2e407539e74522906ec4d530a6dbc0', 'request_id': 'req-18ecf0c5-a959-4ca5-9d81-9c258afe255d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:00.567877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-18ecf0c5-a959-4ca5-9d81-9c258afe255d admin admin] The token cache is being invalidate because project fe2e407539e74522906ec4d530a6dbc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:28:00.568154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-18ecf0c5-a959-4ca5-9d81-9c258afe255d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:00.568916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-18ecf0c5-a959-4ca5-9d81-9c258afe255d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:00.570469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1124/2240] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:28:00 2022] DELETE /identity/v3/projects/fe2e407539e74522906ec4d530a6dbc0 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:00.577271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c8519-6a77-46da-9779-18e99ef5430b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:00.578016 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c8519-6a77-46da-9779-18e99ef5430b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:00.578485 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c8519-6a77-46da-9779-18e99ef5430b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:00.608456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fa6c8519-6a77-46da-9779-18e99ef5430b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:00.609727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fa6c8519-6a77-46da-9779-18e99ef5430b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:00.736332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1117/2241] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:00.745885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c920fb-2570-4440-96fe-08f584b25a1b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:00.746203 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c920fb-2570-4440-96fe-08f584b25a1b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:00.746555 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c920fb-2570-4440-96fe-08f584b25a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:00.776382 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-37c920fb-2570-4440-96fe-08f584b25a1b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:00.777731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-37c920fb-2570-4440-96fe-08f584b25a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:00.792660 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1125/2242] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:00.798416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c8519-6a77-46da-9779-18e99ef5430b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:00.813685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c8519-6a77-46da-9779-18e99ef5430b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:00.822886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1948031-441b-46d7-b27e-56a0116d42da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:00.824118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1948031-441b-46d7-b27e-56a0116d42da admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:00.824511 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1948031-441b-46d7-b27e-56a0116d42da admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:00.824926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1948031-441b-46d7-b27e-56a0116d42da admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:00.826057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1948031-441b-46d7-b27e-56a0116d42da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:00.827000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1948031-441b-46d7-b27e-56a0116d42da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:00.829394 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1948031-441b-46d7-b27e-56a0116d42da admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:00.862967 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1118/2243] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:28:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:28:00.869604 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c920fb-2570-4440-96fe-08f584b25a1b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:00.885244 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c920fb-2570-4440-96fe-08f584b25a1b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:00.893756 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8351a7-8f7e-403a-85dd-7a4b2c1f4195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:00.894428 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8351a7-8f7e-403a-85dd-7a4b2c1f4195 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:00.895284 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8351a7-8f7e-403a-85dd-7a4b2c1f4195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:00.895284 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8351a7-8f7e-403a-85dd-7a4b2c1f4195 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:00.895668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8351a7-8f7e-403a-85dd-7a4b2c1f4195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:00.897807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8351a7-8f7e-403a-85dd-7a4b2c1f4195 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:00.984514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1126/2244] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:28:00 2022] POST /identity/v3/projects => generated 367 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:01.003742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1948031-441b-46d7-b27e-56a0116d42da admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:01.051905 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1948031-441b-46d7-b27e-56a0116d42da admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:01.203853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:01.286857 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03995a66-6d65-49c0-af2d-fa6c9941820b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:01.287869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03995a66-6d65-49c0-af2d-fa6c9941820b admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:01.288154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03995a66-6d65-49c0-af2d-fa6c9941820b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:01.288428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03995a66-6d65-49c0-af2d-fa6c9941820b admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:01.289069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03995a66-6d65-49c0-af2d-fa6c9941820b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:01.293027 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03995a66-6d65-49c0-af2d-fa6c9941820b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:01.391923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1119/2245] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:28:00 2022] POST /identity/v3/users => generated 284 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:01.395408 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8351a7-8f7e-403a-85dd-7a4b2c1f4195 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:01.435400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8351a7-8f7e-403a-85dd-7a4b2c1f4195 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:01.454162 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18debc8b-b773-4883-a4f4-859d11d3447d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:01.454718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18debc8b-b773-4883-a4f4-859d11d3447d admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:01.455081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18debc8b-b773-4883-a4f4-859d11d3447d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:01.455360 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18debc8b-b773-4883-a4f4-859d11d3447d admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:01.456032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18debc8b-b773-4883-a4f4-859d11d3447d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:01.457664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18debc8b-b773-4883-a4f4-859d11d3447d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:01.479558 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1127/2246] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:01.483428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03995a66-6d65-49c0-af2d-fa6c9941820b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:01.509296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03995a66-6d65-49c0-af2d-fa6c9941820b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:01.522171 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d346b-a855-4090-a945-af3bdf24e382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:01.522171 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d346b-a855-4090-a945-af3bdf24e382 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:01.522787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d346b-a855-4090-a945-af3bdf24e382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:01.523211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d346b-a855-4090-a945-af3bdf24e382 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:01.524041 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d346b-a855-4090-a945-af3bdf24e382 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:01.529608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d346b-a855-4090-a945-af3bdf24e382 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:01.571015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1120/2247] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:01 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:01.575325 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18debc8b-b773-4883-a4f4-859d11d3447d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:01.588094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18debc8b-b773-4883-a4f4-859d11d3447d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:01.600095 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8504f08-8806-46dd-8ea6-b70253ab2717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:01.600847 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8504f08-8806-46dd-8ea6-b70253ab2717 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:01.601185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8504f08-8806-46dd-8ea6-b70253ab2717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:01.601605 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8504f08-8806-46dd-8ea6-b70253ab2717 admin admin] PATH_INFO: `/v3/projects/794df556d4984788ac6fad4218dd04d4/users/895feac52fb74e3bbf71d26a70634c59/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:01.631800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8504f08-8806-46dd-8ea6-b70253ab2717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=794df556d4984788ac6fad4218dd04d4, user_id=895feac52fb74e3bbf71d26a70634c59, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:01.635068 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8504f08-8806-46dd-8ea6-b70253ab2717 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:01.652531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1128/2248] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:28:01 2022] PUT /identity/v3/projects/794df556d4984788ac6fad4218dd04d4/users/895feac52fb74e3bbf71d26a70634c59/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:28:01.665076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a56c68-215d-4ca4-81b6-af2a8883d572 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:01.666006 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a56c68-215d-4ca4-81b6-af2a8883d572 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:01.666315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a56c68-215d-4ca4-81b6-af2a8883d572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:01.701831 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a2a56c68-215d-4ca4-81b6-af2a8883d572 None None] MFA Rules not processed for user `895feac52fb74e3bbf71d26a70634c59`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:01.703686 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a2a56c68-215d-4ca4-81b6-af2a8883d572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:01.828197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1121/2249] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:01.834938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8504f08-8806-46dd-8ea6-b70253ab2717 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:01.992020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8504f08-8806-46dd-8ea6-b70253ab2717 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:02.002336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec744d96-f12e-4700-9144-f8092986b22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:02.002787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec744d96-f12e-4700-9144-f8092986b22e admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:02.003066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec744d96-f12e-4700-9144-f8092986b22e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:02.003486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec744d96-f12e-4700-9144-f8092986b22e admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:02.004138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec744d96-f12e-4700-9144-f8092986b22e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:02.006081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec744d96-f12e-4700-9144-f8092986b22e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:02.013746 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:02.049365 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1129/2250] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:28:01 2022] POST /identity/v3/projects => generated 365 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:02.053672 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a56c68-215d-4ca4-81b6-af2a8883d572 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:02.069664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a56c68-215d-4ca4-81b6-af2a8883d572 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:02.182681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02afb5-44d8-4f1b-b268-880c36a01ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:02.183701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02afb5-44d8-4f1b-b268-880c36a01ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:02.183974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02afb5-44d8-4f1b-b268-880c36a01ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:02.184943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02afb5-44d8-4f1b-b268-880c36a01ff8 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:02.186060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02afb5-44d8-4f1b-b268-880c36a01ff8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:02.188189 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02afb5-44d8-4f1b-b268-880c36a01ff8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:02.218403 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1122/2251] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:28:02 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:28:02.224346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec744d96-f12e-4700-9144-f8092986b22e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:02.237826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec744d96-f12e-4700-9144-f8092986b22e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:02.244906 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba11416-fcf1-48f5-a589-6e44c348c6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:02.245964 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba11416-fcf1-48f5-a589-6e44c348c6ce admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:02.246421 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba11416-fcf1-48f5-a589-6e44c348c6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:02.247578 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba11416-fcf1-48f5-a589-6e44c348c6ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:02.249107 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba11416-fcf1-48f5-a589-6e44c348c6ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:02.251483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba11416-fcf1-48f5-a589-6e44c348c6ce admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:02.287664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1130/2252] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:02.292741 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02afb5-44d8-4f1b-b268-880c36a01ff8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:02.324147 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02afb5-44d8-4f1b-b268-880c36a01ff8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:02.351076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2bf57-1117-452e-aac2-ba527bf6c481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:02.351945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2bf57-1117-452e-aac2-ba527bf6c481 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:02.352240 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2bf57-1117-452e-aac2-ba527bf6c481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:02.352521 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2bf57-1117-452e-aac2-ba527bf6c481 admin admin] PATH_INFO: `/v3/projects/ffb87d4c45ae43959a30922966dbb6d1/users/685f18adf2854a3a91abdb7cd6fadca8/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:02.380788 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2bf57-1117-452e-aac2-ba527bf6c481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb87d4c45ae43959a30922966dbb6d1, user_id=685f18adf2854a3a91abdb7cd6fadca8, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:02.386531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a2bf57-1117-452e-aac2-ba527bf6c481 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:02.410656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1123/2253] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:28:02 2022] PUT /identity/v3/projects/ffb87d4c45ae43959a30922966dbb6d1/users/685f18adf2854a3a91abdb7cd6fadca8/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:02.418359 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba11416-fcf1-48f5-a589-6e44c348c6ce admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:02.436645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba11416-fcf1-48f5-a589-6e44c348c6ce admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:02.557805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83397c5b-3813-4b50-8838-38895795db23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:02.558851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83397c5b-3813-4b50-8838-38895795db23 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:02.559309 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83397c5b-3813-4b50-8838-38895795db23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:02.559759 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83397c5b-3813-4b50-8838-38895795db23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:02.561117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83397c5b-3813-4b50-8838-38895795db23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:02.563378 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83397c5b-3813-4b50-8838-38895795db23 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:02.584662 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2bf57-1117-452e-aac2-ba527bf6c481 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:02.585227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1131/2254] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:02 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:02.598883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2bf57-1117-452e-aac2-ba527bf6c481 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:02.616034 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298abad2-24df-49a6-beba-1ec9f11c2e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:02.617690 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298abad2-24df-49a6-beba-1ec9f11c2e77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:02.618243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298abad2-24df-49a6-beba-1ec9f11c2e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:02.620807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298abad2-24df-49a6-beba-1ec9f11c2e77 admin admin] PATH_INFO: `/v3/projects/ffb87d4c45ae43959a30922966dbb6d1/users/685f18adf2854a3a91abdb7cd6fadca8/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:02.624271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298abad2-24df-49a6-beba-1ec9f11c2e77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb87d4c45ae43959a30922966dbb6d1, user_id=685f18adf2854a3a91abdb7cd6fadca8, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:02.638573 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298abad2-24df-49a6-beba-1ec9f11c2e77 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:02.645708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:02.670372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1124/2255] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:28:02 2022] PUT /identity/v3/projects/ffb87d4c45ae43959a30922966dbb6d1/users/685f18adf2854a3a91abdb7cd6fadca8/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:02.684055 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec403d-46a5-48db-b8cb-330c00943ca2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:02.692764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec403d-46a5-48db-b8cb-330c00943ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:02.697241 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec403d-46a5-48db-b8cb-330c00943ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:02.746924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-3cec403d-46a5-48db-b8cb-330c00943ca2 None None] MFA Rules not processed for user `685f18adf2854a3a91abdb7cd6fadca8`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:02.749451 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3cec403d-46a5-48db-b8cb-330c00943ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:02.968280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1132/2256] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:02 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:03.007640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298abad2-24df-49a6-beba-1ec9f11c2e77 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:03.011540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-298abad2-24df-49a6-beba-1ec9f11c2e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:03.163987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298abad2-24df-49a6-beba-1ec9f11c2e77 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:03.191503 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4dad7b-fdf6-4815-8c18-95f8434ce205 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:03.193028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4dad7b-fdf6-4815-8c18-95f8434ce205 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:03.194017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4dad7b-fdf6-4815-8c18-95f8434ce205 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:03.195044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4dad7b-fdf6-4815-8c18-95f8434ce205 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:03.226453 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4dad7b-fdf6-4815-8c18-95f8434ce205 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:03.229517 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4dad7b-fdf6-4815-8c18-95f8434ce205 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:03.274313 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1125/2257] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:28:03.396193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec403d-46a5-48db-b8cb-330c00943ca2 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:03.409340 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec403d-46a5-48db-b8cb-330c00943ca2 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:03.413457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd09e7f-3388-484a-9ed5-7d7faf3ba682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:03.414139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd09e7f-3388-484a-9ed5-7d7faf3ba682 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:03.414442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd09e7f-3388-484a-9ed5-7d7faf3ba682 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:03.414716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd09e7f-3388-484a-9ed5-7d7faf3ba682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:03.519078 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:03.529533 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd09e7f-3388-484a-9ed5-7d7faf3ba682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:03.532887 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd09e7f-3388-484a-9ed5-7d7faf3ba682 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:03.553331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1133/2258] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:03 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:04.014604 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4dad7b-fdf6-4815-8c18-95f8434ce205 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:04.025207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5c4dad7b-fdf6-4815-8c18-95f8434ce205 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:04.117586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4dad7b-fdf6-4815-8c18-95f8434ce205 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:04.122365 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21d24c-5ef0-4111-8161-a63e3b2ac707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:04.122936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21d24c-5ef0-4111-8161-a63e3b2ac707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:04.123208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21d24c-5ef0-4111-8161-a63e3b2ac707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:04.123470 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d21d24c-5ef0-4111-8161-a63e3b2ac707 admin admin] PATH_INFO: `/v3/users/ac3c334ba8794f579c13a177520e29aa` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:04.124605 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21d24c-5ef0-4111-8161-a63e3b2ac707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac3c334ba8794f579c13a177520e29aa)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:04.126575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d21d24c-5ef0-4111-8161-a63e3b2ac707 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:04.188033 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1d21d24c-5ef0-4111-8161-a63e3b2ac707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac3c334ba8794f579c13a177520e29aa'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:04.189067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1d21d24c-5ef0-4111-8161-a63e3b2ac707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac3c334ba8794f579c13a177520e29aa'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:04.203777 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1d21d24c-5ef0-4111-8161-a63e3b2ac707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac3c334ba8794f579c13a177520e29aa'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:04.218811 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1d21d24c-5ef0-4111-8161-a63e3b2ac707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac3c334ba8794f579c13a177520e29aa'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:04.219486 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1d21d24c-5ef0-4111-8161-a63e3b2ac707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac3c334ba8794f579c13a177520e29aa'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:04.233709 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1126/2259] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:28:04 2022] DELETE /identity/v3/users/ac3c334ba8794f579c13a177520e29aa => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:04.238516 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b35be6-cc54-4007-8aa6-d16703ec27ac None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:04.239020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b35be6-cc54-4007-8aa6-d16703ec27ac None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:04.239618 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b35be6-cc54-4007-8aa6-d16703ec27ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:04.271334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-95b35be6-cc54-4007-8aa6-d16703ec27ac None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:04.272890 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-95b35be6-cc54-4007-8aa6-d16703ec27ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:04.370672 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1134/2260] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:28:04.382141 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21d24c-5ef0-4111-8161-a63e3b2ac707 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:04.384346 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1d21d24c-5ef0-4111-8161-a63e3b2ac707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:04.435749 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:04.509469 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d21d24c-5ef0-4111-8161-a63e3b2ac707 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:04.519115 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5876dd8-56d5-4306-8d12-05e383bcb9c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:04.519818 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5876dd8-56d5-4306-8d12-05e383bcb9c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:04.520163 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5876dd8-56d5-4306-8d12-05e383bcb9c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:04.520489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5876dd8-56d5-4306-8d12-05e383bcb9c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:04.533995 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5876dd8-56d5-4306-8d12-05e383bcb9c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:04.535982 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5876dd8-56d5-4306-8d12-05e383bcb9c1 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:04.550580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1127/2261] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:04.963844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b35be6-cc54-4007-8aa6-d16703ec27ac None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:04.965660 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-95b35be6-cc54-4007-8aa6-d16703ec27ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:05.000906 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b35be6-cc54-4007-8aa6-d16703ec27ac None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:05.007852 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b5691c-89fd-4f63-8571-39bc57c485e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:05.008621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5691c-89fd-4f63-8571-39bc57c485e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:05.008995 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5691c-89fd-4f63-8571-39bc57c485e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:05.009370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b5691c-89fd-4f63-8571-39bc57c485e1 admin admin] PATH_INFO: `/v3/projects/40cadfb2bb8e4f5cbf604d9ce2887d90` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:05.010718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b5691c-89fd-4f63-8571-39bc57c485e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40cadfb2bb8e4f5cbf604d9ce2887d90)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:05.012970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b5691c-89fd-4f63-8571-39bc57c485e1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:05.082336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b8b5691c-89fd-4f63-8571-39bc57c485e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40cadfb2bb8e4f5cbf604d9ce2887d90', 'request_id': 'req-b8b5691c-89fd-4f63-8571-39bc57c485e1'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:05.089834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b8b5691c-89fd-4f63-8571-39bc57c485e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40cadfb2bb8e4f5cbf604d9ce2887d90', 'request_id': 'req-b8b5691c-89fd-4f63-8571-39bc57c485e1'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:05.104256 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b8b5691c-89fd-4f63-8571-39bc57c485e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40cadfb2bb8e4f5cbf604d9ce2887d90', 'request_id': 'req-b8b5691c-89fd-4f63-8571-39bc57c485e1'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:05.130547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b8b5691c-89fd-4f63-8571-39bc57c485e1 admin admin] The token cache is being invalidate because project 40cadfb2bb8e4f5cbf604d9ce2887d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:28:05.130547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b8b5691c-89fd-4f63-8571-39bc57c485e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:05.130547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b8b5691c-89fd-4f63-8571-39bc57c485e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:05.131058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1135/2262] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:28:04 2022] DELETE /identity/v3/projects/40cadfb2bb8e4f5cbf604d9ce2887d90 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:05.139564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbba457-f462-4656-bc47-4105284dd4b7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:05.140169 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbba457-f462-4656-bc47-4105284dd4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:05.140169 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbba457-f462-4656-bc47-4105284dd4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:05.166932 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-8fbba457-f462-4656-bc47-4105284dd4b7 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:05.168261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-8fbba457-f462-4656-bc47-4105284dd4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:05.294181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1128/2263] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:05.298245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a6432-a2d9-46d2-a6a4-720f9887ef3f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:05.298576 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a6432-a2d9-46d2-a6a4-720f9887ef3f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:05.298981 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a6432-a2d9-46d2-a6a4-720f9887ef3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:05.345300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b46a6432-a2d9-46d2-a6a4-720f9887ef3f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:05.348036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b46a6432-a2d9-46d2-a6a4-720f9887ef3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:05.378954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbba457-f462-4656-bc47-4105284dd4b7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:05.388084 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1136/2264] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:28:05.408047 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbba457-f462-4656-bc47-4105284dd4b7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:05.420029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a73ba4-4cde-44e7-806e-36cd4b7d1cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:05.421053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a73ba4-4cde-44e7-806e-36cd4b7d1cc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:05.421513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a73ba4-4cde-44e7-806e-36cd4b7d1cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:05.421983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a73ba4-4cde-44e7-806e-36cd4b7d1cc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:05.423384 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a73ba4-4cde-44e7-806e-36cd4b7d1cc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:05.423915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a73ba4-4cde-44e7-806e-36cd4b7d1cc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:05.436990 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a73ba4-4cde-44e7-806e-36cd4b7d1cc8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:05.464826 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1129/2265] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:28:05.472951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a6432-a2d9-46d2-a6a4-720f9887ef3f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:05.511716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a6432-a2d9-46d2-a6a4-720f9887ef3f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:05.540507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f31860-9ba2-4763-8a4b-b87803d1b877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:05.541251 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f31860-9ba2-4763-8a4b-b87803d1b877 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:05.541749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f31860-9ba2-4763-8a4b-b87803d1b877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:05.542335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f31860-9ba2-4763-8a4b-b87803d1b877 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:05.543319 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f31860-9ba2-4763-8a4b-b87803d1b877 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:05.546474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f31860-9ba2-4763-8a4b-b87803d1b877 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:05.586512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1137/2266] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:28:05 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:05.592971 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a73ba4-4cde-44e7-806e-36cd4b7d1cc8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:05.611540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a73ba4-4cde-44e7-806e-36cd4b7d1cc8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:05.747525 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba1a31-7dc4-4812-b88d-f17d0e5e85d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:05.752201 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba1a31-7dc4-4812-b88d-f17d0e5e85d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:05.752486 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba1a31-7dc4-4812-b88d-f17d0e5e85d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:05.752765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aba1a31-7dc4-4812-b88d-f17d0e5e85d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:05.753458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba1a31-7dc4-4812-b88d-f17d0e5e85d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:05.756644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aba1a31-7dc4-4812-b88d-f17d0e5e85d2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:05.828582 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1130/2267] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:28:05 2022] POST /identity/v3/users => generated 286 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:28:05.832904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f31860-9ba2-4763-8a4b-b87803d1b877 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:05.869578 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f31860-9ba2-4763-8a4b-b87803d1b877 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:05.876376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a941f7-6e91-4573-be46-8194776456ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:05.876608 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a941f7-6e91-4573-be46-8194776456ca admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:05.876855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a941f7-6e91-4573-be46-8194776456ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:05.877127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a941f7-6e91-4573-be46-8194776456ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:05.877998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a941f7-6e91-4573-be46-8194776456ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:05.880592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a941f7-6e91-4573-be46-8194776456ca admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:05.895716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1138/2268] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:05.900965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba1a31-7dc4-4812-b88d-f17d0e5e85d2 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:05.923675 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aba1a31-7dc4-4812-b88d-f17d0e5e85d2 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:05.933201 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b800c1-155f-46f2-8d6b-52b1029bf1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:05.933847 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b800c1-155f-46f2-8d6b-52b1029bf1ad admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:05.934423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b800c1-155f-46f2-8d6b-52b1029bf1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:05.934765 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b800c1-155f-46f2-8d6b-52b1029bf1ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:05.935568 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b800c1-155f-46f2-8d6b-52b1029bf1ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:05.937873 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b800c1-155f-46f2-8d6b-52b1029bf1ad admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:05.963228 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1131/2269] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:05.964118 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a941f7-6e91-4573-be46-8194776456ca admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:05.993445 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a941f7-6e91-4573-be46-8194776456ca admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:06.008818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525cb06f-eb5f-42e1-b0be-b3a6836949cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:06.010300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cb06f-eb5f-42e1-b0be-b3a6836949cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:06.011013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cb06f-eb5f-42e1-b0be-b3a6836949cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:06.011707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525cb06f-eb5f-42e1-b0be-b3a6836949cd admin admin] PATH_INFO: `/v3/projects/7292443f39f548e79c3bb9d07bccb0df/users/22ba5db8353847d7b14ba2d4cb391073/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:06.046705 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cb06f-eb5f-42e1-b0be-b3a6836949cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7292443f39f548e79c3bb9d07bccb0df, user_id=22ba5db8353847d7b14ba2d4cb391073, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:06.052929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525cb06f-eb5f-42e1-b0be-b3a6836949cd admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:06.094036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1139/2270] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:28:05 2022] PUT /identity/v3/projects/7292443f39f548e79c3bb9d07bccb0df/users/22ba5db8353847d7b14ba2d4cb391073/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:28:06.101087 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25a563-16d5-4b94-a2dc-d0b6081c2649 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:06.101583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25a563-16d5-4b94-a2dc-d0b6081c2649 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:06.102069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e25a563-16d5-4b94-a2dc-d0b6081c2649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:06.135787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3e25a563-16d5-4b94-a2dc-d0b6081c2649 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:06.138556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3e25a563-16d5-4b94-a2dc-d0b6081c2649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:06.179801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:06.258636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1132/2271] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:06.274888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525cb06f-eb5f-42e1-b0be-b3a6836949cd admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:06.278039 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-525cb06f-eb5f-42e1-b0be-b3a6836949cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:06.321402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:06.425998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525cb06f-eb5f-42e1-b0be-b3a6836949cd admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:06.433257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ee998-868c-44e3-96f1-36ffc34cc6b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:06.433882 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ee998-868c-44e3-96f1-36ffc34cc6b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:06.434151 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ee998-868c-44e3-96f1-36ffc34cc6b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:06.435157 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565ee998-868c-44e3-96f1-36ffc34cc6b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:06.468253 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ee998-868c-44e3-96f1-36ffc34cc6b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:06.471385 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565ee998-868c-44e3-96f1-36ffc34cc6b0 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:06.486149 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1140/2272] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:08.518328 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfde50-2e16-4bca-aa4f-be8f634745c3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:08.519754 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfde50-2e16-4bca-aa4f-be8f634745c3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:08.520598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bfde50-2e16-4bca-aa4f-be8f634745c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:08.584828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d2bfde50-2e16-4bca-aa4f-be8f634745c3 None None] MFA Rules not processed for user `22ba5db8353847d7b14ba2d4cb391073`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:08.587022 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d2bfde50-2e16-4bca-aa4f-be8f634745c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:08.696313 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1133/2273] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:08.712580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ee998-868c-44e3-96f1-36ffc34cc6b0 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:08.714343 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-565ee998-868c-44e3-96f1-36ffc34cc6b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:08.794447 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565ee998-868c-44e3-96f1-36ffc34cc6b0 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:08.798871 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b65ef-1673-4fb9-9076-117aefa7cd77 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:08.799758 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b65ef-1673-4fb9-9076-117aefa7cd77 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:08.800168 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b65ef-1673-4fb9-9076-117aefa7cd77 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:08.800743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b65ef-1673-4fb9-9076-117aefa7cd77 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:08.812405 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b65ef-1673-4fb9-9076-117aefa7cd77 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:08.814676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b65ef-1673-4fb9-9076-117aefa7cd77 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:08.828333 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1141/2274] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:10.847719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bfde50-2e16-4bca-aa4f-be8f634745c3 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:10.849196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d2bfde50-2e16-4bca-aa4f-be8f634745c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:10.864143 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bfde50-2e16-4bca-aa4f-be8f634745c3 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:10.870810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a506a560-f8d0-44a7-a8a3-2bc54a51cbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:10.870810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506a560-f8d0-44a7-a8a3-2bc54a51cbb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:10.870810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506a560-f8d0-44a7-a8a3-2bc54a51cbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:10.870810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a506a560-f8d0-44a7-a8a3-2bc54a51cbb0 admin admin] PATH_INFO: `/v3/users/b7a4e434c3c74b239c50da6b5d500c23` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:10.871245 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506a560-f8d0-44a7-a8a3-2bc54a51cbb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a4e434c3c74b239c50da6b5d500c23)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:10.873077 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a506a560-f8d0-44a7-a8a3-2bc54a51cbb0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:10.929992 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a506a560-f8d0-44a7-a8a3-2bc54a51cbb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a4e434c3c74b239c50da6b5d500c23'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:10.930908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a506a560-f8d0-44a7-a8a3-2bc54a51cbb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a4e434c3c74b239c50da6b5d500c23'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:10.943624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a506a560-f8d0-44a7-a8a3-2bc54a51cbb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a4e434c3c74b239c50da6b5d500c23'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:10.956696 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a506a560-f8d0-44a7-a8a3-2bc54a51cbb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a4e434c3c74b239c50da6b5d500c23'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:10.958064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a506a560-f8d0-44a7-a8a3-2bc54a51cbb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a4e434c3c74b239c50da6b5d500c23'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:10.974984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1134/2275] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:28:10 2022] DELETE /identity/v3/users/b7a4e434c3c74b239c50da6b5d500c23 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:11.171012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b65ef-1673-4fb9-9076-117aefa7cd77 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:11.188307 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b65ef-1673-4fb9-9076-117aefa7cd77 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:11.252624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadb375-cf18-4c01-9a9d-3e614790f7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:11.253256 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadb375-cf18-4c01-9a9d-3e614790f7f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:11.253529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadb375-cf18-4c01-9a9d-3e614790f7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:11.253796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadb375-cf18-4c01-9a9d-3e614790f7f3 admin admin] PATH_INFO: `/v3/projects/fd3f4fde69fc4507837f9feb27ceed01` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:11.254935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aadb375-cf18-4c01-9a9d-3e614790f7f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd3f4fde69fc4507837f9feb27ceed01)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:11.256869 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aadb375-cf18-4c01-9a9d-3e614790f7f3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:11.306568 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6aadb375-cf18-4c01-9a9d-3e614790f7f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd3f4fde69fc4507837f9feb27ceed01', 'request_id': 'req-6aadb375-cf18-4c01-9a9d-3e614790f7f3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:11.313429 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6aadb375-cf18-4c01-9a9d-3e614790f7f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd3f4fde69fc4507837f9feb27ceed01', 'request_id': 'req-6aadb375-cf18-4c01-9a9d-3e614790f7f3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:11.321337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6aadb375-cf18-4c01-9a9d-3e614790f7f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd3f4fde69fc4507837f9feb27ceed01', 'request_id': 'req-6aadb375-cf18-4c01-9a9d-3e614790f7f3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:11.322690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:11.337736 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6aadb375-cf18-4c01-9a9d-3e614790f7f3 admin admin] The token cache is being invalidate because project fd3f4fde69fc4507837f9feb27ceed01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:28:11.337973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6aadb375-cf18-4c01-9a9d-3e614790f7f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:11.338664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6aadb375-cf18-4c01-9a9d-3e614790f7f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:11.340237 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1142/2276] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:28:11 2022] DELETE /identity/v3/projects/fd3f4fde69fc4507837f9feb27ceed01 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:11.345699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463f300-66f6-462d-9720-8203cb7de11d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:11.346003 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463f300-66f6-462d-9720-8203cb7de11d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:11.346391 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463f300-66f6-462d-9720-8203cb7de11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:11.368556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5463f300-66f6-462d-9720-8203cb7de11d None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:11.370178 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5463f300-66f6-462d-9720-8203cb7de11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:11.404311 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:11.460880 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1135/2277] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:11.468691 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab34abc-b345-4e81-bd11-85be314c5818 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:11.468967 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab34abc-b345-4e81-bd11-85be314c5818 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:11.469241 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab34abc-b345-4e81-bd11-85be314c5818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:11.490210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-eab34abc-b345-4e81-bd11-85be314c5818 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:11.491499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-eab34abc-b345-4e81-bd11-85be314c5818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:11.500972 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1143/2278] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:11.505091 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463f300-66f6-462d-9720-8203cb7de11d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:11.516908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463f300-66f6-462d-9720-8203cb7de11d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:11.521574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a952b-34e2-4915-b17c-556bd18ff51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:11.522186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a952b-34e2-4915-b17c-556bd18ff51f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:11.522493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a952b-34e2-4915-b17c-556bd18ff51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:11.522767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a952b-34e2-4915-b17c-556bd18ff51f admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:11.523421 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a952b-34e2-4915-b17c-556bd18ff51f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:11.523720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a952b-34e2-4915-b17c-556bd18ff51f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:11.525400 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1a952b-34e2-4915-b17c-556bd18ff51f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:11.539271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1136/2279] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:28:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:28:11.546173 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab34abc-b345-4e81-bd11-85be314c5818 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:11.555928 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab34abc-b345-4e81-bd11-85be314c5818 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:11.563671 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b20d2-ef51-492d-b739-4632c6a33263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:11.564288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b20d2-ef51-492d-b739-4632c6a33263 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:11.564568 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b20d2-ef51-492d-b739-4632c6a33263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:11.564840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719b20d2-ef51-492d-b739-4632c6a33263 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:11.565625 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b20d2-ef51-492d-b739-4632c6a33263 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:11.567468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719b20d2-ef51-492d-b739-4632c6a33263 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:11.603916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1144/2280] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:28:11 2022] POST /identity/v3/projects => generated 393 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:28:11.608826 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a952b-34e2-4915-b17c-556bd18ff51f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:11.623981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1a952b-34e2-4915-b17c-556bd18ff51f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:11.697572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14543484-4e61-4243-a179-621e9909a73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:11.698197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14543484-4e61-4243-a179-621e9909a73d admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:11.699148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14543484-4e61-4243-a179-621e9909a73d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:11.699148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14543484-4e61-4243-a179-621e9909a73d admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:11.699407 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14543484-4e61-4243-a179-621e9909a73d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:11.701347 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14543484-4e61-4243-a179-621e9909a73d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:11.727835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1137/2281] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:28:11 2022] POST /identity/v3/users => generated 297 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:11.734934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b20d2-ef51-492d-b739-4632c6a33263 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:11.748814 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719b20d2-ef51-492d-b739-4632c6a33263 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:11.753816 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aad91f-56e0-4db7-b6fc-559d532e0d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:11.754461 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aad91f-56e0-4db7-b6fc-559d532e0d9a admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:11.754887 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aad91f-56e0-4db7-b6fc-559d532e0d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:11.755161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aad91f-56e0-4db7-b6fc-559d532e0d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:11.755811 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aad91f-56e0-4db7-b6fc-559d532e0d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:11.757598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aad91f-56e0-4db7-b6fc-559d532e0d9a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:11.768486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1145/2282] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:11.772545 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14543484-4e61-4243-a179-621e9909a73d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:11.783250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14543484-4e61-4243-a179-621e9909a73d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:11.788075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8eea04-486b-4ac8-9c09-d912cbdb048d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:11.788757 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8eea04-486b-4ac8-9c09-d912cbdb048d admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:11.789102 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8eea04-486b-4ac8-9c09-d912cbdb048d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:11.789439 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8eea04-486b-4ac8-9c09-d912cbdb048d admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:11.790151 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8eea04-486b-4ac8-9c09-d912cbdb048d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:11.791844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8eea04-486b-4ac8-9c09-d912cbdb048d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:11.804364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1138/2283] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:11.811612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aad91f-56e0-4db7-b6fc-559d532e0d9a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:11.829049 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aad91f-56e0-4db7-b6fc-559d532e0d9a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:11.837233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ea5ae-c9f6-4247-bfcd-bcf62379f5e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:11.838384 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ea5ae-c9f6-4247-bfcd-bcf62379f5e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:11.838882 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ea5ae-c9f6-4247-bfcd-bcf62379f5e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:11.839375 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495ea5ae-c9f6-4247-bfcd-bcf62379f5e5 admin admin] PATH_INFO: `/v3/projects/ce23bc924f714d72ad20d32a3f5b6d25/users/461b9a3987744067a62234aaa8d086d0/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:11.869172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ea5ae-c9f6-4247-bfcd-bcf62379f5e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce23bc924f714d72ad20d32a3f5b6d25, user_id=461b9a3987744067a62234aaa8d086d0, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:11.874718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495ea5ae-c9f6-4247-bfcd-bcf62379f5e5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:11.891734 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1146/2284] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:28:11 2022] PUT /identity/v3/projects/ce23bc924f714d72ad20d32a3f5b6d25/users/461b9a3987744067a62234aaa8d086d0/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:11.897811 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d3445-c522-4a0d-90b3-112b920655c9 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:11.898609 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d3445-c522-4a0d-90b3-112b920655c9 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:11.898609 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5d3445-c522-4a0d-90b3-112b920655c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:11.919303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1a5d3445-c522-4a0d-90b3-112b920655c9 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:11.920511 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1a5d3445-c522-4a0d-90b3-112b920655c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:11.984130 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1139/2285] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:28:11.995788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ea5ae-c9f6-4247-bfcd-bcf62379f5e5 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:11.998143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-495ea5ae-c9f6-4247-bfcd-bcf62379f5e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:12.056027 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495ea5ae-c9f6-4247-bfcd-bcf62379f5e5 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:12.063069 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79293842-98eb-4e1b-828e-3f7188ab6094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:12.063664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79293842-98eb-4e1b-828e-3f7188ab6094 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:12.063939 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79293842-98eb-4e1b-828e-3f7188ab6094 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:12.064208 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79293842-98eb-4e1b-828e-3f7188ab6094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:12.073727 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79293842-98eb-4e1b-828e-3f7188ab6094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:12.075731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79293842-98eb-4e1b-828e-3f7188ab6094 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:12.085062 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1147/2286] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:15.117002 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d3445-c522-4a0d-90b3-112b920655c9 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:15.118607 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1a5d3445-c522-4a0d-90b3-112b920655c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:15.131251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a5d3445-c522-4a0d-90b3-112b920655c9 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:15.136305 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59003c5f-ef29-4efc-8ec9-4e2a65c1c843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:15.137670 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59003c5f-ef29-4efc-8ec9-4e2a65c1c843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:15.138043 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59003c5f-ef29-4efc-8ec9-4e2a65c1c843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:15.138451 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59003c5f-ef29-4efc-8ec9-4e2a65c1c843 admin admin] PATH_INFO: `/v3/users/895feac52fb74e3bbf71d26a70634c59` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:15.139756 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59003c5f-ef29-4efc-8ec9-4e2a65c1c843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=895feac52fb74e3bbf71d26a70634c59)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:15.142041 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59003c5f-ef29-4efc-8ec9-4e2a65c1c843 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:15.185541 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:15.206489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-59003c5f-ef29-4efc-8ec9-4e2a65c1c843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '895feac52fb74e3bbf71d26a70634c59'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:15.207286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-59003c5f-ef29-4efc-8ec9-4e2a65c1c843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '895feac52fb74e3bbf71d26a70634c59'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:15.223448 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-59003c5f-ef29-4efc-8ec9-4e2a65c1c843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '895feac52fb74e3bbf71d26a70634c59'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:15.233349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-59003c5f-ef29-4efc-8ec9-4e2a65c1c843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '895feac52fb74e3bbf71d26a70634c59'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:15.234259 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-59003c5f-ef29-4efc-8ec9-4e2a65c1c843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '895feac52fb74e3bbf71d26a70634c59'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:15.249807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1140/2287] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:28:15 2022] DELETE /identity/v3/users/895feac52fb74e3bbf71d26a70634c59 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:15.254024 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42ab93-2589-4398-9197-dab344cd6d4f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:15.254552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42ab93-2589-4398-9197-dab344cd6d4f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:15.254630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42ab93-2589-4398-9197-dab344cd6d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:15.276550 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-be42ab93-2589-4398-9197-dab344cd6d4f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:15.277852 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-be42ab93-2589-4398-9197-dab344cd6d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:15.353920 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1148/2288] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:15.363245 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59003c5f-ef29-4efc-8ec9-4e2a65c1c843 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:15.364457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-59003c5f-ef29-4efc-8ec9-4e2a65c1c843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:15.445538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59003c5f-ef29-4efc-8ec9-4e2a65c1c843 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:15.449748 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df851608-5e0a-4d52-bc0e-6ffb1784c870 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:15.451131 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df851608-5e0a-4d52-bc0e-6ffb1784c870 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:15.451663 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df851608-5e0a-4d52-bc0e-6ffb1784c870 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:15.459133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df851608-5e0a-4d52-bc0e-6ffb1784c870 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:15.476402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df851608-5e0a-4d52-bc0e-6ffb1784c870 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:15.477864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df851608-5e0a-4d52-bc0e-6ffb1784c870 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:15.489823 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1141/2289] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:15.651293 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afc8d5-effb-4e71-a0a3-a690c408fe9c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:15.651897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afc8d5-effb-4e71-a0a3-a690c408fe9c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:15.652323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afc8d5-effb-4e71-a0a3-a690c408fe9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:15.676777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d0afc8d5-effb-4e71-a0a3-a690c408fe9c None None] MFA Rules not processed for user `22ba5db8353847d7b14ba2d4cb391073`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:15.679075 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d0afc8d5-effb-4e71-a0a3-a690c408fe9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:15.739045 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:15.822355 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1149/2290] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:16.429958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df851608-5e0a-4d52-bc0e-6ffb1784c870 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:16.434333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-df851608-5e0a-4d52-bc0e-6ffb1784c870 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:16.472117 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df851608-5e0a-4d52-bc0e-6ffb1784c870 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:16.488550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63662560-df95-4aea-a696-82e83b8d5b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:16.493684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63662560-df95-4aea-a696-82e83b8d5b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:16.494156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63662560-df95-4aea-a696-82e83b8d5b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:16.494722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63662560-df95-4aea-a696-82e83b8d5b83 admin admin] PATH_INFO: `/v3/projects/794df556d4984788ac6fad4218dd04d4` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:16.497292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63662560-df95-4aea-a696-82e83b8d5b83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=794df556d4984788ac6fad4218dd04d4)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:16.501777 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63662560-df95-4aea-a696-82e83b8d5b83 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:16.614560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-63662560-df95-4aea-a696-82e83b8d5b83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '794df556d4984788ac6fad4218dd04d4', 'request_id': 'req-63662560-df95-4aea-a696-82e83b8d5b83'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:16.627979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-63662560-df95-4aea-a696-82e83b8d5b83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '794df556d4984788ac6fad4218dd04d4', 'request_id': 'req-63662560-df95-4aea-a696-82e83b8d5b83'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:16.643275 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-63662560-df95-4aea-a696-82e83b8d5b83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '794df556d4984788ac6fad4218dd04d4', 'request_id': 'req-63662560-df95-4aea-a696-82e83b8d5b83'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:16.666518 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-63662560-df95-4aea-a696-82e83b8d5b83 admin admin] The token cache is being invalidate because project 794df556d4984788ac6fad4218dd04d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:28:16.667401 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-63662560-df95-4aea-a696-82e83b8d5b83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:16.668991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-63662560-df95-4aea-a696-82e83b8d5b83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:16.672598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1142/2291] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:28:16 2022] DELETE /identity/v3/projects/794df556d4984788ac6fad4218dd04d4 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:16.676448 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afc8d5-effb-4e71-a0a3-a690c408fe9c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:16.691514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afc8d5-effb-4e71-a0a3-a690c408fe9c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:16.837739 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df2e3a-f95c-4a7e-a8d7-2095d7402266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:16.838976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df2e3a-f95c-4a7e-a8d7-2095d7402266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:16.839448 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df2e3a-f95c-4a7e-a8d7-2095d7402266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:16.839899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2df2e3a-f95c-4a7e-a8d7-2095d7402266 admin admin] PATH_INFO: `/v3/users/685f18adf2854a3a91abdb7cd6fadca8` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:16.842469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df2e3a-f95c-4a7e-a8d7-2095d7402266 admin admin] RBAC: Authorizing `identity:delete_user(user_id=685f18adf2854a3a91abdb7cd6fadca8)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:16.845001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2df2e3a-f95c-4a7e-a8d7-2095d7402266 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:16.944216 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c2df2e3a-f95c-4a7e-a8d7-2095d7402266 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '685f18adf2854a3a91abdb7cd6fadca8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:16.959476 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c2df2e3a-f95c-4a7e-a8d7-2095d7402266 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '685f18adf2854a3a91abdb7cd6fadca8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:16.980509 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c2df2e3a-f95c-4a7e-a8d7-2095d7402266 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '685f18adf2854a3a91abdb7cd6fadca8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:16.982376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c2df2e3a-f95c-4a7e-a8d7-2095d7402266 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '685f18adf2854a3a91abdb7cd6fadca8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:16.999776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c2df2e3a-f95c-4a7e-a8d7-2095d7402266 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '685f18adf2854a3a91abdb7cd6fadca8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:17.011791 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1150/2292] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:28:16 2022] DELETE /identity/v3/users/685f18adf2854a3a91abdb7cd6fadca8 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:17.878531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63662560-df95-4aea-a696-82e83b8d5b83 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:17.881914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-63662560-df95-4aea-a696-82e83b8d5b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:17.901944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63662560-df95-4aea-a696-82e83b8d5b83 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:17.986852 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:18.058890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ab1bd-28d9-4c62-99fa-7813f820b642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:18.060076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ab1bd-28d9-4c62-99fa-7813f820b642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:18.060520 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ab1bd-28d9-4c62-99fa-7813f820b642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:18.060935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ab1bd-28d9-4c62-99fa-7813f820b642 admin admin] PATH_INFO: `/v3/projects/ffb87d4c45ae43959a30922966dbb6d1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:18.062411 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ab1bd-28d9-4c62-99fa-7813f820b642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffb87d4c45ae43959a30922966dbb6d1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:18.064840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ab1bd-28d9-4c62-99fa-7813f820b642 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:18.073773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df2e3a-f95c-4a7e-a8d7-2095d7402266 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:18.077629 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c2df2e3a-f95c-4a7e-a8d7-2095d7402266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:18.097789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2df2e3a-f95c-4a7e-a8d7-2095d7402266 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:18.115561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a167d1-7861-45f3-b03f-9bbe9fff3474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:18.116652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a167d1-7861-45f3-b03f-9bbe9fff3474 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:18.117247 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a167d1-7861-45f3-b03f-9bbe9fff3474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:18.117766 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a167d1-7861-45f3-b03f-9bbe9fff3474 admin admin] PATH_INFO: `/v3/users/1250a8d8b1494e07b48e0515f70e865c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:18.120014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a167d1-7861-45f3-b03f-9bbe9fff3474 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1250a8d8b1494e07b48e0515f70e865c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:18.127240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a167d1-7861-45f3-b03f-9bbe9fff3474 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:18.194352 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6a5ab1bd-28d9-4c62-99fa-7813f820b642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffb87d4c45ae43959a30922966dbb6d1', 'request_id': 'req-6a5ab1bd-28d9-4c62-99fa-7813f820b642'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:18.211948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6a5ab1bd-28d9-4c62-99fa-7813f820b642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffb87d4c45ae43959a30922966dbb6d1', 'request_id': 'req-6a5ab1bd-28d9-4c62-99fa-7813f820b642'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:18.240818 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6a5ab1bd-28d9-4c62-99fa-7813f820b642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffb87d4c45ae43959a30922966dbb6d1', 'request_id': 'req-6a5ab1bd-28d9-4c62-99fa-7813f820b642'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:18.259551 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c9a167d1-7861-45f3-b03f-9bbe9fff3474 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1250a8d8b1494e07b48e0515f70e865c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:18.261594 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:18.269503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c9a167d1-7861-45f3-b03f-9bbe9fff3474 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1250a8d8b1494e07b48e0515f70e865c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:18.276739 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6a5ab1bd-28d9-4c62-99fa-7813f820b642 admin admin] The token cache is being invalidate because project ffb87d4c45ae43959a30922966dbb6d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:28:18.277039 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6a5ab1bd-28d9-4c62-99fa-7813f820b642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:18.279513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6a5ab1bd-28d9-4c62-99fa-7813f820b642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:18.281411 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1143/2293] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:28:17 2022] DELETE /identity/v3/projects/ffb87d4c45ae43959a30922966dbb6d1 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:18.286792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e23d75-6285-4de7-a546-070a3bb10e9b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:18.287252 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e23d75-6285-4de7-a546-070a3bb10e9b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:18.287883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e23d75-6285-4de7-a546-070a3bb10e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:18.300012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c9a167d1-7861-45f3-b03f-9bbe9fff3474 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1250a8d8b1494e07b48e0515f70e865c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:18.302365 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c9a167d1-7861-45f3-b03f-9bbe9fff3474 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1250a8d8b1494e07b48e0515f70e865c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:18.333594 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-92e23d75-6285-4de7-a546-070a3bb10e9b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:18.335209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-92e23d75-6285-4de7-a546-070a3bb10e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:18.358180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c9a167d1-7861-45f3-b03f-9bbe9fff3474 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1250a8d8b1494e07b48e0515f70e865c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:18.382302 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1151/2294] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:28:18 2022] DELETE /identity/v3/users/1250a8d8b1494e07b48e0515f70e865c => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:18.387011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c97b0-202a-4d1c-85d1-27dde733fdba None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:18.387686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c97b0-202a-4d1c-85d1-27dde733fdba None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:18.388122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c97b0-202a-4d1c-85d1-27dde733fdba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:18.425393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b81c97b0-202a-4d1c-85d1-27dde733fdba None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:18.426924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b81c97b0-202a-4d1c-85d1-27dde733fdba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:18.482937 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1144/2295] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:18.491810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06923b37-ba2a-4384-a974-fc63edc77006 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:18.511349 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06923b37-ba2a-4384-a974-fc63edc77006 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:18.511788 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06923b37-ba2a-4384-a974-fc63edc77006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:18.544333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-06923b37-ba2a-4384-a974-fc63edc77006 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:18.545557 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-06923b37-ba2a-4384-a974-fc63edc77006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:18.567295 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1152/2296] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:18.568698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f1566-e628-4f6b-9ca9-f5acd079860f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:18.568959 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f1566-e628-4f6b-9ca9-f5acd079860f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:18.569277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6f1566-e628-4f6b-9ca9-f5acd079860f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:18.604409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-be6f1566-e628-4f6b-9ca9-f5acd079860f None None] MFA Rules not processed for user `461b9a3987744067a62234aaa8d086d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:18.604409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-be6f1566-e628-4f6b-9ca9-f5acd079860f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:18.651490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1145/2297] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:18.652348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06923b37-ba2a-4384-a974-fc63edc77006 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:18.653665 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-06923b37-ba2a-4384-a974-fc63edc77006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:18.661748 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:18.738715 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06923b37-ba2a-4384-a974-fc63edc77006 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:18.742907 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19614db-2c8f-4f08-b963-a8e5ae8870d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:18.743520 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19614db-2c8f-4f08-b963-a8e5ae8870d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:18.743793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19614db-2c8f-4f08-b963-a8e5ae8870d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:18.744058 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19614db-2c8f-4f08-b963-a8e5ae8870d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:18.746454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1153/2298] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:18.748214 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f1566-e628-4f6b-9ca9-f5acd079860f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:18.756264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19614db-2c8f-4f08-b963-a8e5ae8870d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:18.757705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19614db-2c8f-4f08-b963-a8e5ae8870d1 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:18.762561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6f1566-e628-4f6b-9ca9-f5acd079860f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:18.768473 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2083a-c5ab-4f75-8d94-a2f78baefe54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:18.768617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1146/2299] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:18.768835 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2083a-c5ab-4f75-8d94-a2f78baefe54 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:18.769119 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2083a-c5ab-4f75-8d94-a2f78baefe54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:18.771067 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2083a-c5ab-4f75-8d94-a2f78baefe54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:18.771067 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2083a-c5ab-4f75-8d94-a2f78baefe54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:18.771267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2083a-c5ab-4f75-8d94-a2f78baefe54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:18.772755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2083a-c5ab-4f75-8d94-a2f78baefe54 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:18.776908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da634e-70a2-4239-b671-3d7a9556f5b4 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:18.777225 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da634e-70a2-4239-b671-3d7a9556f5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:18.777617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da634e-70a2-4239-b671-3d7a9556f5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:18.799859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1154/2300] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:28:18.801461 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2083a-c5ab-4f75-8d94-a2f78baefe54 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:18.808597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f2da634e-70a2-4239-b671-3d7a9556f5b4 None None] MFA Rules not processed for user `461b9a3987744067a62234aaa8d086d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:18.810849 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f2da634e-70a2-4239-b671-3d7a9556f5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:18.817676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2083a-c5ab-4f75-8d94-a2f78baefe54 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:18.830797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea6f9ba-5144-43d1-9760-770badfcbbda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:18.831053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1147/2301] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:18 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:18.832598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da634e-70a2-4239-b671-3d7a9556f5b4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:18.832951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6f9ba-5144-43d1-9760-770badfcbbda admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:18.834591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f2da634e-70a2-4239-b671-3d7a9556f5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:18.835230 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6f9ba-5144-43d1-9760-770badfcbbda admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:18.836120 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea6f9ba-5144-43d1-9760-770badfcbbda admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:18.837860 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea6f9ba-5144-43d1-9760-770badfcbbda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:18.842756 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea6f9ba-5144-43d1-9760-770badfcbbda admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:18.914873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1155/2302] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:28:18 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:18.926321 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea6f9ba-5144-43d1-9760-770badfcbbda admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:18.930183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da634e-70a2-4239-b671-3d7a9556f5b4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:18.953170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:19.041496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01553e-68c4-48f8-857c-3e772c98c4b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:19.042229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01553e-68c4-48f8-857c-3e772c98c4b1 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:19.042600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01553e-68c4-48f8-857c-3e772c98c4b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:19.042950 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f01553e-68c4-48f8-857c-3e772c98c4b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:19.054706 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea6f9ba-5144-43d1-9760-770badfcbbda admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:19.061466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0df826-2056-430c-8ded-bb67793947fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:19.062093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0df826-2056-430c-8ded-bb67793947fa admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:19.062552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0df826-2056-430c-8ded-bb67793947fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:19.062955 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0df826-2056-430c-8ded-bb67793947fa admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:19.063791 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0df826-2056-430c-8ded-bb67793947fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:19.066703 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0df826-2056-430c-8ded-bb67793947fa admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:19.095684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:19.099601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1156/2303] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:28:18 2022] POST /identity/v3/users => generated 287 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:19.103713 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0df826-2056-430c-8ded-bb67793947fa admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:19.124065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0df826-2056-430c-8ded-bb67793947fa admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:19.131151 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f814daf-240c-41aa-9760-041154fc056b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:19.131578 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01553e-68c4-48f8-857c-3e772c98c4b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:19.132075 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f814daf-240c-41aa-9760-041154fc056b admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:19.132392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f814daf-240c-41aa-9760-041154fc056b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:19.132765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f814daf-240c-41aa-9760-041154fc056b admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:19.133272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f01553e-68c4-48f8-857c-3e772c98c4b1 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:19.133801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f814daf-240c-41aa-9760-041154fc056b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:19.135616 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f814daf-240c-41aa-9760-041154fc056b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:19.147493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1148/2304] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:18 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:28:19.168438 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1157/2305] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:19.169847 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01553e-68c4-48f8-857c-3e772c98c4b1 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:19.172537 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f814daf-240c-41aa-9760-041154fc056b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:19.173807 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4f01553e-68c4-48f8-857c-3e772c98c4b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:19.197281 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f814daf-240c-41aa-9760-041154fc056b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:19.210275 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f01553e-68c4-48f8-857c-3e772c98c4b1 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:19.210607 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9a858-daea-40b0-b257-1daa6ad59ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:19.211312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9a858-daea-40b0-b257-1daa6ad59ef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:19.212109 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9a858-daea-40b0-b257-1daa6ad59ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:19.212879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9a858-daea-40b0-b257-1daa6ad59ef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:19.213556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9a858-daea-40b0-b257-1daa6ad59ef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:19.217618 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9a858-daea-40b0-b257-1daa6ad59ef0 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:19.218496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcef641-5e72-4060-9b37-9789eda8cb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:19.220303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcef641-5e72-4060-9b37-9789eda8cb31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:19.220809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcef641-5e72-4060-9b37-9789eda8cb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:19.221304 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcef641-5e72-4060-9b37-9789eda8cb31 admin admin] PATH_INFO: `/v3/projects/d12d49c49967427e971bf9d64f93de6a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:19.225103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcef641-5e72-4060-9b37-9789eda8cb31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d12d49c49967427e971bf9d64f93de6a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:19.227206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcef641-5e72-4060-9b37-9789eda8cb31 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:19.243381 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1158/2306] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:19 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:19.248921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9a858-daea-40b0-b257-1daa6ad59ef0 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:19.290135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9a858-daea-40b0-b257-1daa6ad59ef0 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:19.303576 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae156615-dab7-4c0d-a230-821e4c0a9e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:19.304506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae156615-dab7-4c0d-a230-821e4c0a9e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:19.304801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae156615-dab7-4c0d-a230-821e4c0a9e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:19.305090 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae156615-dab7-4c0d-a230-821e4c0a9e32 admin admin] PATH_INFO: `/v3/projects/baf28579f5c545ae9e331f9960b0a044/users/b25b20ff576b48edb8d590aad615ebae/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:19.332840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae156615-dab7-4c0d-a230-821e4c0a9e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=baf28579f5c545ae9e331f9960b0a044, user_id=b25b20ff576b48edb8d590aad615ebae, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:19.333466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-afcef641-5e72-4060-9b37-9789eda8cb31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd12d49c49967427e971bf9d64f93de6a', 'request_id': 'req-afcef641-5e72-4060-9b37-9789eda8cb31'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:19.335531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae156615-dab7-4c0d-a230-821e4c0a9e32 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:19.343291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-afcef641-5e72-4060-9b37-9789eda8cb31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd12d49c49967427e971bf9d64f93de6a', 'request_id': 'req-afcef641-5e72-4060-9b37-9789eda8cb31'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:19.357286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-afcef641-5e72-4060-9b37-9789eda8cb31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd12d49c49967427e971bf9d64f93de6a', 'request_id': 'req-afcef641-5e72-4060-9b37-9789eda8cb31'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:19.372045 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1159/2307] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:28:19 2022] PUT /identity/v3/projects/baf28579f5c545ae9e331f9960b0a044/users/b25b20ff576b48edb8d590aad615ebae/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:19.373910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088c721-19f8-48a6-9aa5-643f555aa185 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:19.374292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088c721-19f8-48a6-9aa5-643f555aa185 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:19.374495 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f088c721-19f8-48a6-9aa5-643f555aa185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:19.381489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-afcef641-5e72-4060-9b37-9789eda8cb31 admin admin] The token cache is being invalidate because project d12d49c49967427e971bf9d64f93de6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:28:19.382246 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-afcef641-5e72-4060-9b37-9789eda8cb31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:19.383380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-afcef641-5e72-4060-9b37-9789eda8cb31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:19.385691 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1149/2308] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:28:19 2022] DELETE /identity/v3/projects/d12d49c49967427e971bf9d64f93de6a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:19.389673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcef641-5e72-4060-9b37-9789eda8cb31 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:19.391591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-afcef641-5e72-4060-9b37-9789eda8cb31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:19.416285 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f088c721-19f8-48a6-9aa5-643f555aa185 None None] MFA Rules not processed for user `b25b20ff576b48edb8d590aad615ebae`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:19.418360 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f088c721-19f8-48a6-9aa5-643f555aa185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:19.496836 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcef641-5e72-4060-9b37-9789eda8cb31 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:19.569247 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1160/2309] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:19.573731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088c721-19f8-48a6-9aa5-643f555aa185 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:19.576868 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f088c721-19f8-48a6-9aa5-643f555aa185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:19.658350 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdbf5cb-2890-4ecd-8718-392c0097940e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:19.659697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbf5cb-2890-4ecd-8718-392c0097940e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:19.660358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbf5cb-2890-4ecd-8718-392c0097940e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:19.662502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdbf5cb-2890-4ecd-8718-392c0097940e admin admin] PATH_INFO: `/v3/users/cd4c15ea1f784edaa0bcf2a5b1133e43` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:19.667459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbf5cb-2890-4ecd-8718-392c0097940e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd4c15ea1f784edaa0bcf2a5b1133e43)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:19.671561 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdbf5cb-2890-4ecd-8718-392c0097940e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:19.687860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:19.715049 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:19.753150 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f088c721-19f8-48a6-9aa5-643f555aa185 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:19.758571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c09081-3edb-45d2-8c4b-4dbaedb453ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:19.759190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c09081-3edb-45d2-8c4b-4dbaedb453ed admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:19.759465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c09081-3edb-45d2-8c4b-4dbaedb453ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:19.759785 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c09081-3edb-45d2-8c4b-4dbaedb453ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:19.760574 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c09081-3edb-45d2-8c4b-4dbaedb453ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:19.762876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c09081-3edb-45d2-8c4b-4dbaedb453ed admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:19.784977 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3fdbf5cb-2890-4ecd-8718-392c0097940e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd4c15ea1f784edaa0bcf2a5b1133e43'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:19.785980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3fdbf5cb-2890-4ecd-8718-392c0097940e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd4c15ea1f784edaa0bcf2a5b1133e43'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:19.800891 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1161/2310] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:28:19 2022] POST /identity/v3/projects => generated 375 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:19.804542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c09081-3edb-45d2-8c4b-4dbaedb453ed admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:19.808063 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3fdbf5cb-2890-4ecd-8718-392c0097940e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd4c15ea1f784edaa0bcf2a5b1133e43'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:19.823182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3fdbf5cb-2890-4ecd-8718-392c0097940e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd4c15ea1f784edaa0bcf2a5b1133e43'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:19.823903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3fdbf5cb-2890-4ecd-8718-392c0097940e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd4c15ea1f784edaa0bcf2a5b1133e43'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:19.857310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1150/2311] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:28:19 2022] DELETE /identity/v3/users/cd4c15ea1f784edaa0bcf2a5b1133e43 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:28:19.859918 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303325f-9af4-43dc-a418-a674becc2d49 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:19.874180 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303325f-9af4-43dc-a418-a674becc2d49 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:19.874406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303325f-9af4-43dc-a418-a674becc2d49 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:19.878027 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1151/2312] 10.222.0.76 () {54 vars in 912 bytes} [Wed Aug 24 15:28:19 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:28:19.884181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cabc2c-bad8-4512-85e1-166d77d7e1e4 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:19.884471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cabc2c-bad8-4512-85e1-166d77d7e1e4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:19.884748 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cabc2c-bad8-4512-85e1-166d77d7e1e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:19.930953 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-56cabc2c-bad8-4512-85e1-166d77d7e1e4 None None] MFA Rules not processed for user `5427bd0ac88042338b956c10cfd07baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:19.930953 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-56cabc2c-bad8-4512-85e1-166d77d7e1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:19.948704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c09081-3edb-45d2-8c4b-4dbaedb453ed admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:20.033867 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1152/2313] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 15:28:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:28:20.054958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cabc2c-bad8-4512-85e1-166d77d7e1e4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:20.056471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-56cabc2c-bad8-4512-85e1-166d77d7e1e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:20.070720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cabc2c-bad8-4512-85e1-166d77d7e1e4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:20.078705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f138d7-382b-4473-b9b2-3f8b2ad19050 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:20.079474 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f138d7-382b-4473-b9b2-3f8b2ad19050 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:20.080034 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f138d7-382b-4473-b9b2-3f8b2ad19050 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:20.080376 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f138d7-382b-4473-b9b2-3f8b2ad19050 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:20.103875 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f138d7-382b-4473-b9b2-3f8b2ad19050 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:20.104797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3053e7-1c15-42b8-9e5a-a0e40e37d786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:20.106638 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3053e7-1c15-42b8-9e5a-a0e40e37d786 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:20.107394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3053e7-1c15-42b8-9e5a-a0e40e37d786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:20.107884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3053e7-1c15-42b8-9e5a-a0e40e37d786 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:20.109018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3053e7-1c15-42b8-9e5a-a0e40e37d786 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:20.110706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f138d7-382b-4473-b9b2-3f8b2ad19050 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:20.113532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3053e7-1c15-42b8-9e5a-a0e40e37d786 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:20.133644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1153/2314] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:28:20.162391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1162/2315] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:28:19 2022] POST /identity/v3/users => generated 294 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:28:20.165078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f138d7-382b-4473-b9b2-3f8b2ad19050 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:20.167115 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e2f138d7-382b-4473-b9b2-3f8b2ad19050 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:20.194010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f138d7-382b-4473-b9b2-3f8b2ad19050 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:20.206987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2159f-ba25-4a40-9dd3-753e66d40aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:20.207731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2159f-ba25-4a40-9dd3-753e66d40aec admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:20.208014 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2159f-ba25-4a40-9dd3-753e66d40aec admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:20.208319 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa2159f-ba25-4a40-9dd3-753e66d40aec admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:20.209289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2159f-ba25-4a40-9dd3-753e66d40aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:20.219625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa2159f-ba25-4a40-9dd3-753e66d40aec admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:20.248933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3053e7-1c15-42b8-9e5a-a0e40e37d786 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:20.249599 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1154/2316] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:20.282894 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3053e7-1c15-42b8-9e5a-a0e40e37d786 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:20.307698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5cd45-d33e-4210-a1b4-63c9e009dd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:20.312784 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5cd45-d33e-4210-a1b4-63c9e009dd52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:20.313794 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5cd45-d33e-4210-a1b4-63c9e009dd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:20.315118 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c5cd45-d33e-4210-a1b4-63c9e009dd52 admin admin] PATH_INFO: `/v3/projects/8997a329f5c04f89b00d8e876aec1610/users/7eb55e9a624646319d45a8feea8aef97/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:20.379413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5cd45-d33e-4210-a1b4-63c9e009dd52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8997a329f5c04f89b00d8e876aec1610, user_id=7eb55e9a624646319d45a8feea8aef97, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:20.388946 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c5cd45-d33e-4210-a1b4-63c9e009dd52 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:20.450482 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1163/2317] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:28:20 2022] PUT /identity/v3/projects/8997a329f5c04f89b00d8e876aec1610/users/7eb55e9a624646319d45a8feea8aef97/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:20.459643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2159f-ba25-4a40-9dd3-753e66d40aec admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:20.502285 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa2159f-ba25-4a40-9dd3-753e66d40aec admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:20.625597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ef9943-c038-4627-b85f-7321db9184ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:20.627350 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef9943-c038-4627-b85f-7321db9184ab admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:20.628042 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef9943-c038-4627-b85f-7321db9184ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:20.629882 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ef9943-c038-4627-b85f-7321db9184ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:20.631286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef9943-c038-4627-b85f-7321db9184ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:20.635910 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ef9943-c038-4627-b85f-7321db9184ab admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:20.670310 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5cd45-d33e-4210-a1b4-63c9e009dd52 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:20.671120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1155/2318] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:20 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:20.688543 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c5cd45-d33e-4210-a1b4-63c9e009dd52 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:20.704055 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c510f9-88b3-452a-b8a0-94e57ead3df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:20.704770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c510f9-88b3-452a-b8a0-94e57ead3df0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:20.705052 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c510f9-88b3-452a-b8a0-94e57ead3df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:20.705330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c510f9-88b3-452a-b8a0-94e57ead3df0 admin admin] PATH_INFO: `/v3/projects/8997a329f5c04f89b00d8e876aec1610/users/7eb55e9a624646319d45a8feea8aef97/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:20.708210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c510f9-88b3-452a-b8a0-94e57ead3df0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8997a329f5c04f89b00d8e876aec1610, user_id=7eb55e9a624646319d45a8feea8aef97, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:20.712685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c510f9-88b3-452a-b8a0-94e57ead3df0 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:20.745857 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1164/2319] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:28:20 2022] PUT /identity/v3/projects/8997a329f5c04f89b00d8e876aec1610/users/7eb55e9a624646319d45a8feea8aef97/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:20.758870 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9612377-05a3-424d-8918-8062effae120 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:20.759247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9612377-05a3-424d-8918-8062effae120 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:20.759589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9612377-05a3-424d-8918-8062effae120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:20.773578 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c510f9-88b3-452a-b8a0-94e57ead3df0 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:20.775483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b9c510f9-88b3-452a-b8a0-94e57ead3df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:20.803452 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:20.856164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:20.871199 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c9612377-05a3-424d-8918-8062effae120 None None] MFA Rules not processed for user `7eb55e9a624646319d45a8feea8aef97`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:20.872961 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c9612377-05a3-424d-8918-8062effae120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:20.927453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c510f9-88b3-452a-b8a0-94e57ead3df0 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:20.935907 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9ff02-23bc-40db-9a0c-7b75ac8e3d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:20.936696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9ff02-23bc-40db-9a0c-7b75ac8e3d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:20.936977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9ff02-23bc-40db-9a0c-7b75ac8e3d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:20.937253 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9ff02-23bc-40db-9a0c-7b75ac8e3d35 admin admin] PATH_INFO: `/v3/projects/4e522b2f25554e7b8971641de39d9091` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:20.938726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9ff02-23bc-40db-9a0c-7b75ac8e3d35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e522b2f25554e7b8971641de39d9091)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:20.941008 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9ff02-23bc-40db-9a0c-7b75ac8e3d35 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:21.002949 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1156/2320] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:21.018181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9612377-05a3-424d-8918-8062effae120 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:21.021169 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c9612377-05a3-424d-8918-8062effae120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:21.044458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-71b9ff02-23bc-40db-9a0c-7b75ac8e3d35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e522b2f25554e7b8971641de39d9091', 'request_id': 'req-71b9ff02-23bc-40db-9a0c-7b75ac8e3d35'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:21.064867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-71b9ff02-23bc-40db-9a0c-7b75ac8e3d35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e522b2f25554e7b8971641de39d9091', 'request_id': 'req-71b9ff02-23bc-40db-9a0c-7b75ac8e3d35'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:21.083372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-71b9ff02-23bc-40db-9a0c-7b75ac8e3d35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e522b2f25554e7b8971641de39d9091', 'request_id': 'req-71b9ff02-23bc-40db-9a0c-7b75ac8e3d35'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:21.109607 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-71b9ff02-23bc-40db-9a0c-7b75ac8e3d35 admin admin] The token cache is being invalidate because project 4e522b2f25554e7b8971641de39d9091 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:28:21.112237 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-71b9ff02-23bc-40db-9a0c-7b75ac8e3d35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:21.113127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-71b9ff02-23bc-40db-9a0c-7b75ac8e3d35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:21.116107 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1165/2321] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:28:20 2022] DELETE /identity/v3/projects/4e522b2f25554e7b8971641de39d9091 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:28:21.126149 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583485b-2343-45a5-ae29-57bc246df26a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:21.126438 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583485b-2343-45a5-ae29-57bc246df26a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:21.127553 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e583485b-2343-45a5-ae29-57bc246df26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:21.191795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e583485b-2343-45a5-ae29-57bc246df26a None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:21.192561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e583485b-2343-45a5-ae29-57bc246df26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:21.199467 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9612377-05a3-424d-8918-8062effae120 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:21.310275 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1166/2322] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:21.320339 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbfdec-53ac-4c2d-9026-0ced49943339 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:21.320538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a05ca-f0bc-463d-be88-276749637c53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:21.320969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbfdec-53ac-4c2d-9026-0ced49943339 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:21.321073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bbfdec-53ac-4c2d-9026-0ced49943339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:21.321159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a05ca-f0bc-463d-be88-276749637c53 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:21.321430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a05ca-f0bc-463d-be88-276749637c53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:21.321704 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a05ca-f0bc-463d-be88-276749637c53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:21.324019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-da9a05ca-f0bc-463d-be88-276749637c53 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:21.343161 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-18bbfdec-53ac-4c2d-9026-0ced49943339 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:21.344571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-18bbfdec-53ac-4c2d-9026-0ced49943339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:21.363076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1167/2323] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:21.365238 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbfdec-53ac-4c2d-9026-0ced49943339 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:21.375462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bbfdec-53ac-4c2d-9026-0ced49943339 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:21.380740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b697f52-89fb-44cd-b0e5-50a030f2cef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:21.381070 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b697f52-89fb-44cd-b0e5-50a030f2cef1 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:21.381359 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b697f52-89fb-44cd-b0e5-50a030f2cef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:21.381633 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b697f52-89fb-44cd-b0e5-50a030f2cef1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:21.382343 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b697f52-89fb-44cd-b0e5-50a030f2cef1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:21.382702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b697f52-89fb-44cd-b0e5-50a030f2cef1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:21.385288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b697f52-89fb-44cd-b0e5-50a030f2cef1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:21.398118 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1168/2324] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:28:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:28:21.403358 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b697f52-89fb-44cd-b0e5-50a030f2cef1 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:21.414440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b697f52-89fb-44cd-b0e5-50a030f2cef1 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:21.420099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbba48-d366-47f1-b3c6-6b0cc2ac568b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:21.420731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbba48-d366-47f1-b3c6-6b0cc2ac568b admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:21.420998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbba48-d366-47f1-b3c6-6b0cc2ac568b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:21.421306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecbba48-d366-47f1-b3c6-6b0cc2ac568b admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:21.421904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbba48-d366-47f1-b3c6-6b0cc2ac568b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:21.424391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecbba48-d366-47f1-b3c6-6b0cc2ac568b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:21.470345 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1169/2325] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:28:21 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:21.473619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a05ca-f0bc-463d-be88-276749637c53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:21.474796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbba48-d366-47f1-b3c6-6b0cc2ac568b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:21.475700 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a05ca-f0bc-463d-be88-276749637c53 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:21.487531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1157/2326] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:21.533311 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a97bee-0297-4651-89e9-d02c865eb21f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:21.533770 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a97bee-0297-4651-89e9-d02c865eb21f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:21.534321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a97bee-0297-4651-89e9-d02c865eb21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:21.545430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:21.584040 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a0a97bee-0297-4651-89e9-d02c865eb21f None None] MFA Rules not processed for user `461b9a3987744067a62234aaa8d086d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:21.586477 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a0a97bee-0297-4651-89e9-d02c865eb21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:21.598529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:21.678434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecbba48-d366-47f1-b3c6-6b0cc2ac568b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:21.683037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1158/2327] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:21 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:28:21.697369 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9defd-2413-4685-a54d-98c612c7f9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:21.697967 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9defd-2413-4685-a54d-98c612c7f9ae admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:21.700624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9defd-2413-4685-a54d-98c612c7f9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:21.700932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d9defd-2413-4685-a54d-98c612c7f9ae admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:21.701931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9defd-2413-4685-a54d-98c612c7f9ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:21.709222 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d9defd-2413-4685-a54d-98c612c7f9ae admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:21.745929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1170/2328] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:28:21 2022] POST /identity/v3/users => generated 289 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:21.747235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a97bee-0297-4651-89e9-d02c865eb21f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:21.764126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a97bee-0297-4651-89e9-d02c865eb21f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:21.769053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774636c-cfba-4ede-8598-f9ed40bf3cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:21.769817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774636c-cfba-4ede-8598-f9ed40bf3cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:21.770188 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774636c-cfba-4ede-8598-f9ed40bf3cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:21.770641 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c774636c-cfba-4ede-8598-f9ed40bf3cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:21.771423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774636c-cfba-4ede-8598-f9ed40bf3cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:21.773121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c774636c-cfba-4ede-8598-f9ed40bf3cc3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:21.803621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1159/2329] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:28:21.807400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9defd-2413-4685-a54d-98c612c7f9ae admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:21.827006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d9defd-2413-4685-a54d-98c612c7f9ae admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:21.839210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7a919-2ebf-44b8-b005-3307759968c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:21.839930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7a919-2ebf-44b8-b005-3307759968c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:21.841401 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7a919-2ebf-44b8-b005-3307759968c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:21.841679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7a919-2ebf-44b8-b005-3307759968c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:21.843900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7a919-2ebf-44b8-b005-3307759968c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:21.847145 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7a919-2ebf-44b8-b005-3307759968c2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:21.875673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1171/2330] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:21 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:28:21.880402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774636c-cfba-4ede-8598-f9ed40bf3cc3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:21.895977 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c774636c-cfba-4ede-8598-f9ed40bf3cc3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:21.909747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b7101-79aa-489c-98c8-c1fd86232c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:21.912673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b7101-79aa-489c-98c8-c1fd86232c3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:21.913126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b7101-79aa-489c-98c8-c1fd86232c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:21.913830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675b7101-79aa-489c-98c8-c1fd86232c3c admin admin] PATH_INFO: `/v3/projects/d3d7a835b9994a7e833a40ee6208f0e9/users/43880a5b93a14a38907c232a7ce22407/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:21.944705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b7101-79aa-489c-98c8-c1fd86232c3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3d7a835b9994a7e833a40ee6208f0e9, user_id=43880a5b93a14a38907c232a7ce22407, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:21.961207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675b7101-79aa-489c-98c8-c1fd86232c3c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:22.017691 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1160/2331] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:28:21 2022] PUT /identity/v3/projects/d3d7a835b9994a7e833a40ee6208f0e9/users/43880a5b93a14a38907c232a7ce22407/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:22.021132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6deec-d29a-4ff2-a0da-9bdcb5065f5d None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:22.021711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6deec-d29a-4ff2-a0da-9bdcb5065f5d None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:22.022345 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb6deec-d29a-4ff2-a0da-9bdcb5065f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:22.101613 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cfb6deec-d29a-4ff2-a0da-9bdcb5065f5d None None] MFA Rules not processed for user `43880a5b93a14a38907c232a7ce22407`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:22.103267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cfb6deec-d29a-4ff2-a0da-9bdcb5065f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:22.334788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1172/2332] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:28:22.352026 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b7101-79aa-489c-98c8-c1fd86232c3c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:22.354020 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-675b7101-79aa-489c-98c8-c1fd86232c3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:22.464777 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675b7101-79aa-489c-98c8-c1fd86232c3c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:22.481949 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f0c7a3-5c81-4b48-886c-f79895822ce8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:22.484202 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0c7a3-5c81-4b48-886c-f79895822ce8 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:22.485012 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0c7a3-5c81-4b48-886c-f79895822ce8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:22.485952 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f0c7a3-5c81-4b48-886c-f79895822ce8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:22.504203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f0c7a3-5c81-4b48-886c-f79895822ce8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:22.510814 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f0c7a3-5c81-4b48-886c-f79895822ce8 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:22.533311 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1161/2333] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:24.575795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6deec-d29a-4ff2-a0da-9bdcb5065f5d None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:24.580025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cfb6deec-d29a-4ff2-a0da-9bdcb5065f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:24.692169 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb6deec-d29a-4ff2-a0da-9bdcb5065f5d None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:24.698633 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c7003-b60e-4ee7-9e70-dad402664108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:24.700062 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c7003-b60e-4ee7-9e70-dad402664108 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:24.700496 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c7003-b60e-4ee7-9e70-dad402664108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:24.700868 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4c7003-b60e-4ee7-9e70-dad402664108 admin admin] PATH_INFO: `/v3/users/b25b20ff576b48edb8d590aad615ebae` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:24.703282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c7003-b60e-4ee7-9e70-dad402664108 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b25b20ff576b48edb8d590aad615ebae)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:24.705534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4c7003-b60e-4ee7-9e70-dad402664108 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:24.781990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:24.800904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9c4c7003-b60e-4ee7-9e70-dad402664108 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b25b20ff576b48edb8d590aad615ebae'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:24.813293 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9c4c7003-b60e-4ee7-9e70-dad402664108 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b25b20ff576b48edb8d590aad615ebae'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:24.841453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9c4c7003-b60e-4ee7-9e70-dad402664108 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b25b20ff576b48edb8d590aad615ebae'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:24.844993 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9c4c7003-b60e-4ee7-9e70-dad402664108 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b25b20ff576b48edb8d590aad615ebae'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:24.864142 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9c4c7003-b60e-4ee7-9e70-dad402664108 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b25b20ff576b48edb8d590aad615ebae'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:24.883306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1173/2334] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:28:24 2022] DELETE /identity/v3/users/b25b20ff576b48edb8d590aad615ebae => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:24.885006 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e27e13-0e34-405d-95af-d636aa032ca5 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:24.885536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e27e13-0e34-405d-95af-d636aa032ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:24.885536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e27e13-0e34-405d-95af-d636aa032ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:24.927884 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e2e27e13-0e34-405d-95af-d636aa032ca5 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:24.930083 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e2e27e13-0e34-405d-95af-d636aa032ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:25.088948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1162/2335] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:25.149319 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c7003-b60e-4ee7-9e70-dad402664108 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:25.151009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9c4c7003-b60e-4ee7-9e70-dad402664108 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:25.264331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4c7003-b60e-4ee7-9e70-dad402664108 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:25.269370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02346a2-5522-47af-9d04-f039f13c46ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:25.270209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02346a2-5522-47af-9d04-f039f13c46ff service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:25.270579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02346a2-5522-47af-9d04-f039f13c46ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:25.270982 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02346a2-5522-47af-9d04-f039f13c46ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:25.307355 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02346a2-5522-47af-9d04-f039f13c46ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:25.309330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02346a2-5522-47af-9d04-f039f13c46ff service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:25.321932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1174/2336] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:25.769739 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e27e13-0e34-405d-95af-d636aa032ca5 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:25.771471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e2e27e13-0e34-405d-95af-d636aa032ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:25.786403 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e27e13-0e34-405d-95af-d636aa032ca5 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:25.792793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54fac33-1b09-4dce-bb65-1e989b4441c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:25.793982 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fac33-1b09-4dce-bb65-1e989b4441c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:25.794268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fac33-1b09-4dce-bb65-1e989b4441c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:25.794543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54fac33-1b09-4dce-bb65-1e989b4441c0 admin admin] PATH_INFO: `/v3/projects/baf28579f5c545ae9e331f9960b0a044` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:25.795689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54fac33-1b09-4dce-bb65-1e989b4441c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=baf28579f5c545ae9e331f9960b0a044)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:25.797697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54fac33-1b09-4dce-bb65-1e989b4441c0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:25.810974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:25.855047 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c54fac33-1b09-4dce-bb65-1e989b4441c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'baf28579f5c545ae9e331f9960b0a044', 'request_id': 'req-c54fac33-1b09-4dce-bb65-1e989b4441c0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:25.863327 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c54fac33-1b09-4dce-bb65-1e989b4441c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'baf28579f5c545ae9e331f9960b0a044', 'request_id': 'req-c54fac33-1b09-4dce-bb65-1e989b4441c0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:25.873197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c54fac33-1b09-4dce-bb65-1e989b4441c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'baf28579f5c545ae9e331f9960b0a044', 'request_id': 'req-c54fac33-1b09-4dce-bb65-1e989b4441c0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:25.889016 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c54fac33-1b09-4dce-bb65-1e989b4441c0 admin admin] The token cache is being invalidate because project baf28579f5c545ae9e331f9960b0a044 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:28:25.889696 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c54fac33-1b09-4dce-bb65-1e989b4441c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:25.890212 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c54fac33-1b09-4dce-bb65-1e989b4441c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:25.891750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1163/2337] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:28:25 2022] DELETE /identity/v3/projects/baf28579f5c545ae9e331f9960b0a044 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:25.895887 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02346a2-5522-47af-9d04-f039f13c46ff service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:25.907977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02346a2-5522-47af-9d04-f039f13c46ff service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:25.988742 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb5d672-2d91-44ea-bbd9-ad60071371f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:25.989399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5d672-2d91-44ea-bbd9-ad60071371f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:25.989676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5d672-2d91-44ea-bbd9-ad60071371f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:25.989945 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb5d672-2d91-44ea-bbd9-ad60071371f5 admin admin] PATH_INFO: `/v3/users/7eb55e9a624646319d45a8feea8aef97` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:25.991429 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5d672-2d91-44ea-bbd9-ad60071371f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eb55e9a624646319d45a8feea8aef97)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:25.993378 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb5d672-2d91-44ea-bbd9-ad60071371f5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:26.057488 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6bb5d672-2d91-44ea-bbd9-ad60071371f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eb55e9a624646319d45a8feea8aef97'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:26.068147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6bb5d672-2d91-44ea-bbd9-ad60071371f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eb55e9a624646319d45a8feea8aef97'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:26.075082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:26.090473 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6bb5d672-2d91-44ea-bbd9-ad60071371f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eb55e9a624646319d45a8feea8aef97'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:26.093518 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6bb5d672-2d91-44ea-bbd9-ad60071371f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eb55e9a624646319d45a8feea8aef97'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:26.106676 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-6bb5d672-2d91-44ea-bbd9-ad60071371f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eb55e9a624646319d45a8feea8aef97'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:26.113962 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1175/2338] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:28:25 2022] DELETE /identity/v3/users/7eb55e9a624646319d45a8feea8aef97 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:28:26.469699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54fac33-1b09-4dce-bb65-1e989b4441c0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:26.471812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c54fac33-1b09-4dce-bb65-1e989b4441c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:26.507896 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54fac33-1b09-4dce-bb65-1e989b4441c0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:26.566024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385321bb-c4f2-48c7-979f-6a5164e63ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:26.566601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385321bb-c4f2-48c7-979f-6a5164e63ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:26.566876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385321bb-c4f2-48c7-979f-6a5164e63ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:26.567142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385321bb-c4f2-48c7-979f-6a5164e63ddf admin admin] PATH_INFO: `/v3/projects/8997a329f5c04f89b00d8e876aec1610` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:26.568233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385321bb-c4f2-48c7-979f-6a5164e63ddf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8997a329f5c04f89b00d8e876aec1610)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:26.570391 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385321bb-c4f2-48c7-979f-6a5164e63ddf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:26.617536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-385321bb-c4f2-48c7-979f-6a5164e63ddf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8997a329f5c04f89b00d8e876aec1610', 'request_id': 'req-385321bb-c4f2-48c7-979f-6a5164e63ddf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:26.623357 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-385321bb-c4f2-48c7-979f-6a5164e63ddf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8997a329f5c04f89b00d8e876aec1610', 'request_id': 'req-385321bb-c4f2-48c7-979f-6a5164e63ddf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:26.630764 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-385321bb-c4f2-48c7-979f-6a5164e63ddf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8997a329f5c04f89b00d8e876aec1610', 'request_id': 'req-385321bb-c4f2-48c7-979f-6a5164e63ddf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:26.642696 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-385321bb-c4f2-48c7-979f-6a5164e63ddf admin admin] The token cache is being invalidate because project 8997a329f5c04f89b00d8e876aec1610 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:28:26.642982 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-385321bb-c4f2-48c7-979f-6a5164e63ddf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:26.643622 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-385321bb-c4f2-48c7-979f-6a5164e63ddf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:26.645130 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1164/2339] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:28:26 2022] DELETE /identity/v3/projects/8997a329f5c04f89b00d8e876aec1610 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:26.650097 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a510b63-49f1-49a3-b7f7-5ae91c363e30 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:26.650415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a510b63-49f1-49a3-b7f7-5ae91c363e30 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:26.650695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a510b63-49f1-49a3-b7f7-5ae91c363e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:26.673326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8a510b63-49f1-49a3-b7f7-5ae91c363e30 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:26.674591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8a510b63-49f1-49a3-b7f7-5ae91c363e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:26.736374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1176/2340] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:26.744200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28f5a8-5a20-4c63-87dd-2db3618ccc4c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:26.744484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28f5a8-5a20-4c63-87dd-2db3618ccc4c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:26.744756 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c28f5a8-5a20-4c63-87dd-2db3618ccc4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:26.764477 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6c28f5a8-5a20-4c63-87dd-2db3618ccc4c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:26.765680 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6c28f5a8-5a20-4c63-87dd-2db3618ccc4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:26.775134 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1165/2341] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:26.780000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a510b63-49f1-49a3-b7f7-5ae91c363e30 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:26.789961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a510b63-49f1-49a3-b7f7-5ae91c363e30 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:26.795154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a19efe-acd8-49b0-9084-9ad042dab65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:26.795742 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a19efe-acd8-49b0-9084-9ad042dab65c admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:26.796022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a19efe-acd8-49b0-9084-9ad042dab65c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:26.796288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a19efe-acd8-49b0-9084-9ad042dab65c admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:26.796929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a19efe-acd8-49b0-9084-9ad042dab65c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:26.797227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a19efe-acd8-49b0-9084-9ad042dab65c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:26.798845 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a19efe-acd8-49b0-9084-9ad042dab65c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:26.810395 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1177/2342] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:28:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:28:26.814866 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28f5a8-5a20-4c63-87dd-2db3618ccc4c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:26.824641 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c28f5a8-5a20-4c63-87dd-2db3618ccc4c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:26.829350 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea5f49-41ba-4cbd-bff8-263aa5039144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:26.829952 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea5f49-41ba-4cbd-bff8-263aa5039144 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:26.830244 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea5f49-41ba-4cbd-bff8-263aa5039144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:26.830567 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea5f49-41ba-4cbd-bff8-263aa5039144 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:26.831201 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea5f49-41ba-4cbd-bff8-263aa5039144 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:26.833121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea5f49-41ba-4cbd-bff8-263aa5039144 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:26.859079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:26.866437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1166/2343] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:28:26 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:28:26.874800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a19efe-acd8-49b0-9084-9ad042dab65c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:26.886438 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a19efe-acd8-49b0-9084-9ad042dab65c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:26.943201 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319a3ff-5769-46db-bc84-d92d301a7336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:26.943930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319a3ff-5769-46db-bc84-d92d301a7336 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:26.944074 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319a3ff-5769-46db-bc84-d92d301a7336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:26.944349 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1319a3ff-5769-46db-bc84-d92d301a7336 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:26.944993 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319a3ff-5769-46db-bc84-d92d301a7336 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:26.946971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1319a3ff-5769-46db-bc84-d92d301a7336 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:26.970523 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1178/2344] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:28:26 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:26.974946 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea5f49-41ba-4cbd-bff8-263aa5039144 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:26.985435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea5f49-41ba-4cbd-bff8-263aa5039144 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:26.990389 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c19db-5800-4746-a4c5-7d930b4a5ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:26.990984 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c19db-5800-4746-a4c5-7d930b4a5ecc admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:26.991259 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c19db-5800-4746-a4c5-7d930b4a5ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:26.991525 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c19db-5800-4746-a4c5-7d930b4a5ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:26.992162 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c19db-5800-4746-a4c5-7d930b4a5ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:26.993729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c19db-5800-4746-a4c5-7d930b4a5ecc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:27.003704 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1167/2345] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:26 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:27.007644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319a3ff-5769-46db-bc84-d92d301a7336 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:27.018442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1319a3ff-5769-46db-bc84-d92d301a7336 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:27.023129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc62255-95a5-4979-a0f3-aa88afff6646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:27.023724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc62255-95a5-4979-a0f3-aa88afff6646 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:27.024000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc62255-95a5-4979-a0f3-aa88afff6646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:27.024272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc62255-95a5-4979-a0f3-aa88afff6646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:27.024915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc62255-95a5-4979-a0f3-aa88afff6646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:27.026541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc62255-95a5-4979-a0f3-aa88afff6646 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:27.038468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1179/2346] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:27.042909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c19db-5800-4746-a4c5-7d930b4a5ecc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:27.053555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c19db-5800-4746-a4c5-7d930b4a5ecc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:27.058245 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad38b93-6d46-4a90-967c-95da9e948c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:27.058993 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad38b93-6d46-4a90-967c-95da9e948c61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:27.059278 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad38b93-6d46-4a90-967c-95da9e948c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:27.059550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad38b93-6d46-4a90-967c-95da9e948c61 admin admin] PATH_INFO: `/v3/projects/8fcdf2893664457ba53cc8a4793f4d53/users/f5eebe729eee4bf4b41f4de32e5a44c9/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:27.073840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad38b93-6d46-4a90-967c-95da9e948c61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fcdf2893664457ba53cc8a4793f4d53, user_id=f5eebe729eee4bf4b41f4de32e5a44c9, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:27.077198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad38b93-6d46-4a90-967c-95da9e948c61 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:27.091337 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1168/2347] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:28:27 2022] PUT /identity/v3/projects/8fcdf2893664457ba53cc8a4793f4d53/users/f5eebe729eee4bf4b41f4de32e5a44c9/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:28:27.102879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bc9a0-f314-4170-9669-112faef2adc5 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:27.103165 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bc9a0-f314-4170-9669-112faef2adc5 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:27.103437 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07bc9a0-f314-4170-9669-112faef2adc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:27.133687 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f07bc9a0-f314-4170-9669-112faef2adc5 None None] MFA Rules not processed for user `f5eebe729eee4bf4b41f4de32e5a44c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:27.134936 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f07bc9a0-f314-4170-9669-112faef2adc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:27.173261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:27.216132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1180/2348] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:27.227387 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad38b93-6d46-4a90-967c-95da9e948c61 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:27.229087 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5ad38b93-6d46-4a90-967c-95da9e948c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:27.293143 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad38b93-6d46-4a90-967c-95da9e948c61 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:27.296916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e5d2b2-e8d6-4a1f-a76c-35a22020067f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96c71ea88d9a474c9ff7a30555cfa91f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:27.297491 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5d2b2-e8d6-4a1f-a76c-35a22020067f service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:27.297769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5d2b2-e8d6-4a1f-a76c-35a22020067f service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:27.298036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e5d2b2-e8d6-4a1f-a76c-35a22020067f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:27.306351 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5d2b2-e8d6-4a1f-a76c-35a22020067f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:27.307787 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e5d2b2-e8d6-4a1f-a76c-35a22020067f service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:27.316972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1169/2349] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:29.289261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bc9a0-f314-4170-9669-112faef2adc5 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:29.293977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f07bc9a0-f314-4170-9669-112faef2adc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:29.414175 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07bc9a0-f314-4170-9669-112faef2adc5 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:29.427640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19752dc9-c2eb-490c-9bcb-581ff7dbe8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:29.428350 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19752dc9-c2eb-490c-9bcb-581ff7dbe8c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:29.428646 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19752dc9-c2eb-490c-9bcb-581ff7dbe8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:29.428917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19752dc9-c2eb-490c-9bcb-581ff7dbe8c1 admin admin] PATH_INFO: `/v3/users/5cbc40dd72cf466482f5ba3a400b7e21` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:29.430206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19752dc9-c2eb-490c-9bcb-581ff7dbe8c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cbc40dd72cf466482f5ba3a400b7e21)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:29.432396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19752dc9-c2eb-490c-9bcb-581ff7dbe8c1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:29.515284 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-19752dc9-c2eb-490c-9bcb-581ff7dbe8c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cbc40dd72cf466482f5ba3a400b7e21'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:29.526887 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-19752dc9-c2eb-490c-9bcb-581ff7dbe8c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cbc40dd72cf466482f5ba3a400b7e21'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:29.554635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-19752dc9-c2eb-490c-9bcb-581ff7dbe8c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cbc40dd72cf466482f5ba3a400b7e21'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:29.556907 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-19752dc9-c2eb-490c-9bcb-581ff7dbe8c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cbc40dd72cf466482f5ba3a400b7e21'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:29.567177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-19752dc9-c2eb-490c-9bcb-581ff7dbe8c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cbc40dd72cf466482f5ba3a400b7e21'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:29.575214 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1181/2350] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:28:29 2022] DELETE /identity/v3/users/5cbc40dd72cf466482f5ba3a400b7e21 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:29.580078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14e113-4805-4a65-bf1e-0f664f2a47a0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:29.580674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14e113-4805-4a65-bf1e-0f664f2a47a0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:29.581213 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14e113-4805-4a65-bf1e-0f664f2a47a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:29.614180 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3d14e113-4805-4a65-bf1e-0f664f2a47a0 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:29.615618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3d14e113-4805-4a65-bf1e-0f664f2a47a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:29.829002 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1170/2351] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:28:29.849806 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19752dc9-c2eb-490c-9bcb-581ff7dbe8c1 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:29.857569 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-19752dc9-c2eb-490c-9bcb-581ff7dbe8c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:29.954772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19752dc9-c2eb-490c-9bcb-581ff7dbe8c1 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:29.958690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bead73-72d7-450c-b19c-81863a6bac86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:29.959314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bead73-72d7-450c-b19c-81863a6bac86 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:29.959630 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bead73-72d7-450c-b19c-81863a6bac86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:29.960041 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bead73-72d7-450c-b19c-81863a6bac86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:29.969319 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:29.974046 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bead73-72d7-450c-b19c-81863a6bac86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:29.976207 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bead73-72d7-450c-b19c-81863a6bac86 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:29.995210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1182/2352] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:30.730253 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14e113-4805-4a65-bf1e-0f664f2a47a0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:30.732939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3d14e113-4805-4a65-bf1e-0f664f2a47a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:30.748614 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d14e113-4805-4a65-bf1e-0f664f2a47a0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:30.759636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bef412-4363-4d72-a816-69a4bd424172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:30.760579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bef412-4363-4d72-a816-69a4bd424172 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:30.760904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bef412-4363-4d72-a816-69a4bd424172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:30.761441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bef412-4363-4d72-a816-69a4bd424172 admin admin] PATH_INFO: `/v3/projects/d4b182c4a1b0434c9b0e8c8b7e5fd910` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:30.763538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bef412-4363-4d72-a816-69a4bd424172 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d4b182c4a1b0434c9b0e8c8b7e5fd910)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:30.765659 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bef412-4363-4d72-a816-69a4bd424172 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:30.852182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18bef412-4363-4d72-a816-69a4bd424172 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd4b182c4a1b0434c9b0e8c8b7e5fd910', 'request_id': 'req-18bef412-4363-4d72-a816-69a4bd424172'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:30.863379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18bef412-4363-4d72-a816-69a4bd424172 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd4b182c4a1b0434c9b0e8c8b7e5fd910', 'request_id': 'req-18bef412-4363-4d72-a816-69a4bd424172'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:30.869595 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:30.876228 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18bef412-4363-4d72-a816-69a4bd424172 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd4b182c4a1b0434c9b0e8c8b7e5fd910', 'request_id': 'req-18bef412-4363-4d72-a816-69a4bd424172'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:30.894568 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18bef412-4363-4d72-a816-69a4bd424172 admin admin] The token cache is being invalidate because project d4b182c4a1b0434c9b0e8c8b7e5fd910 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:28:30.895165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18bef412-4363-4d72-a816-69a4bd424172 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:30.896193 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18bef412-4363-4d72-a816-69a4bd424172 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:30.904410 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1171/2353] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:28:30 2022] DELETE /identity/v3/projects/d4b182c4a1b0434c9b0e8c8b7e5fd910 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:30.929488 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f427c-c011-4086-851b-2ef07d32d0d3 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:30.931257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f427c-c011-4086-851b-2ef07d32d0d3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:30.931541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08f427c-c011-4086-851b-2ef07d32d0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:30.984261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d08f427c-c011-4086-851b-2ef07d32d0d3 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:30.987874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d08f427c-c011-4086-851b-2ef07d32d0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:31.092688 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1183/2354] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:31.104068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a77fd-06b2-4061-a5c1-89e6cc34e4b7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:31.104533 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a77fd-06b2-4061-a5c1-89e6cc34e4b7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:31.105366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a77fd-06b2-4061-a5c1-89e6cc34e4b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:31.151891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-594a77fd-06b2-4061-a5c1-89e6cc34e4b7 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:31.154572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-594a77fd-06b2-4061-a5c1-89e6cc34e4b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:31.171877 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1172/2355] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:31.179321 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f427c-c011-4086-851b-2ef07d32d0d3 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:31.208737 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08f427c-c011-4086-851b-2ef07d32d0d3 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:31.222874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6578e-9b3a-47f4-ba60-9c4f35cb65a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:31.223918 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6578e-9b3a-47f4-ba60-9c4f35cb65a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:31.224499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6578e-9b3a-47f4-ba60-9c4f35cb65a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:31.224988 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea6578e-9b3a-47f4-ba60-9c4f35cb65a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:31.226314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6578e-9b3a-47f4-ba60-9c4f35cb65a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:31.226864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6578e-9b3a-47f4-ba60-9c4f35cb65a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:31.230264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea6578e-9b3a-47f4-ba60-9c4f35cb65a5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:31.253035 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1184/2356] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:28:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:28:31.259720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a77fd-06b2-4061-a5c1-89e6cc34e4b7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:31.271147 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a77fd-06b2-4061-a5c1-89e6cc34e4b7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:31.277155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1fcc8-3481-4fb7-bf9a-2ef37d713e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:31.278187 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1fcc8-3481-4fb7-bf9a-2ef37d713e29 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:31.278609 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1fcc8-3481-4fb7-bf9a-2ef37d713e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:31.278976 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab1fcc8-3481-4fb7-bf9a-2ef37d713e29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:31.279721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1fcc8-3481-4fb7-bf9a-2ef37d713e29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:31.283339 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab1fcc8-3481-4fb7-bf9a-2ef37d713e29 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:31.317834 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1173/2357] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:28:31 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:28:31.324099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6578e-9b3a-47f4-ba60-9c4f35cb65a5 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:31.341215 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea6578e-9b3a-47f4-ba60-9c4f35cb65a5 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:31.415037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa08aa1-8aae-4148-9321-619a85fdc037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:31.415780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa08aa1-8aae-4148-9321-619a85fdc037 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:31.416062 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa08aa1-8aae-4148-9321-619a85fdc037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:31.416332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa08aa1-8aae-4148-9321-619a85fdc037 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:31.417013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa08aa1-8aae-4148-9321-619a85fdc037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:31.419188 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa08aa1-8aae-4148-9321-619a85fdc037 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:31.455893 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1185/2358] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:28:31 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:31.463923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1fcc8-3481-4fb7-bf9a-2ef37d713e29 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:31.481270 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab1fcc8-3481-4fb7-bf9a-2ef37d713e29 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:31.493869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff09a3-34b4-47b8-98a0-05c35e29b4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:31.494788 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff09a3-34b4-47b8-98a0-05c35e29b4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:31.495272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff09a3-34b4-47b8-98a0-05c35e29b4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:31.495671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cff09a3-34b4-47b8-98a0-05c35e29b4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:31.497276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff09a3-34b4-47b8-98a0-05c35e29b4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:31.499762 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cff09a3-34b4-47b8-98a0-05c35e29b4a6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:31.524053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1174/2359] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:28:31.526260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa08aa1-8aae-4148-9321-619a85fdc037 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:31.539461 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa08aa1-8aae-4148-9321-619a85fdc037 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:31.548444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57d177-ce0e-44bb-852f-075246dfaeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:31.549296 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57d177-ce0e-44bb-852f-075246dfaeb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:31.549806 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57d177-ce0e-44bb-852f-075246dfaeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:31.550082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57d177-ce0e-44bb-852f-075246dfaeb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:31.551080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57d177-ce0e-44bb-852f-075246dfaeb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:31.553154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57d177-ce0e-44bb-852f-075246dfaeb4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:31.575540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1186/2360] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:31 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:31.581886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff09a3-34b4-47b8-98a0-05c35e29b4a6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:31.597727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cff09a3-34b4-47b8-98a0-05c35e29b4a6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:31.609610 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60551d2-9310-40d3-8a69-7e95b2be0aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:31.611467 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60551d2-9310-40d3-8a69-7e95b2be0aa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:31.612166 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60551d2-9310-40d3-8a69-7e95b2be0aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:31.612794 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60551d2-9310-40d3-8a69-7e95b2be0aa0 admin admin] PATH_INFO: `/v3/projects/cba4ef8af239471b9bea1535e6e272af/users/55e7550911424451be8b3bb966f66bd9/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:31.662375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60551d2-9310-40d3-8a69-7e95b2be0aa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba4ef8af239471b9bea1535e6e272af, user_id=55e7550911424451be8b3bb966f66bd9, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:31.662375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60551d2-9310-40d3-8a69-7e95b2be0aa0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:31.688617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1175/2361] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:28:31 2022] PUT /identity/v3/projects/cba4ef8af239471b9bea1535e6e272af/users/55e7550911424451be8b3bb966f66bd9/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:31.700405 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79702d-4095-40e2-9946-de91a69387d5 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:31.700873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79702d-4095-40e2-9946-de91a69387d5 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:31.701357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd79702d-4095-40e2-9946-de91a69387d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:31.739586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cd79702d-4095-40e2-9946-de91a69387d5 None None] MFA Rules not processed for user `55e7550911424451be8b3bb966f66bd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:31.740957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cd79702d-4095-40e2-9946-de91a69387d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:31.799435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:31.841807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1187/2362] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:31.849182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60551d2-9310-40d3-8a69-7e95b2be0aa0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:31.923301 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60551d2-9310-40d3-8a69-7e95b2be0aa0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:31.929015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422f838-f379-4c04-9d78-f53a24e894eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:31.929678 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422f838-f379-4c04-9d78-f53a24e894eb admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:31.929964 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422f838-f379-4c04-9d78-f53a24e894eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:31.930306 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4422f838-f379-4c04-9d78-f53a24e894eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:31.931114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422f838-f379-4c04-9d78-f53a24e894eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:31.934290 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4422f838-f379-4c04-9d78-f53a24e894eb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:31.979120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1176/2363] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:28:31 2022] POST /identity/v3/projects => generated 385 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:31.983465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79702d-4095-40e2-9946-de91a69387d5 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:31.997090 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd79702d-4095-40e2-9946-de91a69387d5 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:32.078768 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4966874-311e-4c29-bf20-7ec0489eca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:32.079801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4966874-311e-4c29-bf20-7ec0489eca71 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:32.080279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4966874-311e-4c29-bf20-7ec0489eca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:32.080744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4966874-311e-4c29-bf20-7ec0489eca71 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:32.081869 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4966874-311e-4c29-bf20-7ec0489eca71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:32.085516 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4966874-311e-4c29-bf20-7ec0489eca71 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:32.115851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1188/2364] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:28:31 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:28:32.119751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422f838-f379-4c04-9d78-f53a24e894eb admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:32.133232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4422f838-f379-4c04-9d78-f53a24e894eb admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:32.142354 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f12d2-cc5d-4137-97e2-26aff824a640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:32.143423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f12d2-cc5d-4137-97e2-26aff824a640 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:32.143924 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f12d2-cc5d-4137-97e2-26aff824a640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:32.144423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8f12d2-cc5d-4137-97e2-26aff824a640 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:32.145614 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f12d2-cc5d-4137-97e2-26aff824a640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:32.148851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8f12d2-cc5d-4137-97e2-26aff824a640 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:32.162330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1177/2365] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:28:32.167138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4966874-311e-4c29-bf20-7ec0489eca71 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:32.178541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4966874-311e-4c29-bf20-7ec0489eca71 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:32.183393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904eb35b-88f5-4ea2-a82c-87c1ab704e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:32.184036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904eb35b-88f5-4ea2-a82c-87c1ab704e2a admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:32.184319 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904eb35b-88f5-4ea2-a82c-87c1ab704e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:32.184593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904eb35b-88f5-4ea2-a82c-87c1ab704e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:32.185232 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904eb35b-88f5-4ea2-a82c-87c1ab704e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:32.186896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904eb35b-88f5-4ea2-a82c-87c1ab704e2a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:32.198905 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1189/2366] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:28:32.205593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f12d2-cc5d-4137-97e2-26aff824a640 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:32.218847 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8f12d2-cc5d-4137-97e2-26aff824a640 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:32.228377 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726888a3-ae16-4010-ac18-4b85902791d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:32.229152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726888a3-ae16-4010-ac18-4b85902791d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:32.229577 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726888a3-ae16-4010-ac18-4b85902791d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:32.229987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726888a3-ae16-4010-ac18-4b85902791d3 admin admin] PATH_INFO: `/v3/projects/d99ccdbf77564700b1c51456dc1d677d/users/2081d751b86a404ba89b4c41c00f386a/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:32.256602 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726888a3-ae16-4010-ac18-4b85902791d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d99ccdbf77564700b1c51456dc1d677d, user_id=2081d751b86a404ba89b4c41c00f386a, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:32.259336 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726888a3-ae16-4010-ac18-4b85902791d3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:32.276082 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1178/2367] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:28:32 2022] PUT /identity/v3/projects/d99ccdbf77564700b1c51456dc1d677d/users/2081d751b86a404ba89b4c41c00f386a/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:28:32.294569 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f6127-d79d-49e2-bd03-d6c54738716b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:32.295064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f6127-d79d-49e2-bd03-d6c54738716b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:32.295553 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33f6127-d79d-49e2-bd03-d6c54738716b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:32.336052 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e33f6127-d79d-49e2-bd03-d6c54738716b None None] MFA Rules not processed for user `2081d751b86a404ba89b4c41c00f386a`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:32.338192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e33f6127-d79d-49e2-bd03-d6c54738716b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:32.444583 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1190/2368] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:28:32.448894 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726888a3-ae16-4010-ac18-4b85902791d3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:32.512112 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:32.548036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726888a3-ae16-4010-ac18-4b85902791d3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:32.556970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0e476-ca43-4002-b5a7-6a57ee8a12dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:32.557833 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0e476-ca43-4002-b5a7-6a57ee8a12dc admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:32.558099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0e476-ca43-4002-b5a7-6a57ee8a12dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:32.559175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc0e476-ca43-4002-b5a7-6a57ee8a12dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:32.560076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0e476-ca43-4002-b5a7-6a57ee8a12dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:32.563035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc0e476-ca43-4002-b5a7-6a57ee8a12dc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:32.600404 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1179/2369] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:28:32 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:32.605321 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f6127-d79d-49e2-bd03-d6c54738716b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:32.617880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33f6127-d79d-49e2-bd03-d6c54738716b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:32.681055 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:32.691148 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b231b7-65e3-44d1-ae51-4c1ec4370bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:32.692656 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b231b7-65e3-44d1-ae51-4c1ec4370bb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:32.693314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b231b7-65e3-44d1-ae51-4c1ec4370bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:32.693476 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b231b7-65e3-44d1-ae51-4c1ec4370bb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:32.694651 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b231b7-65e3-44d1-ae51-4c1ec4370bb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:32.697715 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b231b7-65e3-44d1-ae51-4c1ec4370bb3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:32.744549 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1191/2370] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:28:32 2022] POST /identity/v3/users => generated 299 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:28:32.754288 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0e476-ca43-4002-b5a7-6a57ee8a12dc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:32.770195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc0e476-ca43-4002-b5a7-6a57ee8a12dc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:32.775133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3751c-6c24-4631-b597-5bb27069f246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:32.775749 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3751c-6c24-4631-b597-5bb27069f246 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:32.776026 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3751c-6c24-4631-b597-5bb27069f246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:32.776308 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca3751c-6c24-4631-b597-5bb27069f246 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:32.776951 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3751c-6c24-4631-b597-5bb27069f246 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:32.778598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca3751c-6c24-4631-b597-5bb27069f246 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:32.792294 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1180/2371] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:32.797906 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b231b7-65e3-44d1-ae51-4c1ec4370bb3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:32.809390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b231b7-65e3-44d1-ae51-4c1ec4370bb3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:32.814322 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6621c-439d-472e-9c0e-f4b932786d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:32.815138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6621c-439d-472e-9c0e-f4b932786d5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:32.815426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6621c-439d-472e-9c0e-f4b932786d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:32.815701 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db6621c-439d-472e-9c0e-f4b932786d5e admin admin] PATH_INFO: `/v3/projects/efd6d0d6b8f14f2ea31e438570d0a6cf/users/66fa6728fc894b3da1fbf57ac91316b0/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:32.831678 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6621c-439d-472e-9c0e-f4b932786d5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=efd6d0d6b8f14f2ea31e438570d0a6cf, user_id=66fa6728fc894b3da1fbf57ac91316b0, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:32.834403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db6621c-439d-472e-9c0e-f4b932786d5e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:32.854182 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1192/2372] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:28:32 2022] PUT /identity/v3/projects/efd6d0d6b8f14f2ea31e438570d0a6cf/users/66fa6728fc894b3da1fbf57ac91316b0/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:28:32.861609 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3751c-6c24-4631-b597-5bb27069f246 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:32.890635 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca3751c-6c24-4631-b597-5bb27069f246 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:32.955862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40c650e-d766-4d3e-970c-9db57cedcc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:32.956435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c650e-d766-4d3e-970c-9db57cedcc3a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:32.956669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c650e-d766-4d3e-970c-9db57cedcc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:32.956940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40c650e-d766-4d3e-970c-9db57cedcc3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:32.957721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c650e-d766-4d3e-970c-9db57cedcc3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:32.959596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40c650e-d766-4d3e-970c-9db57cedcc3a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:32.970348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1181/2373] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:32 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:32.974573 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6621c-439d-472e-9c0e-f4b932786d5e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:32.985729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db6621c-439d-472e-9c0e-f4b932786d5e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:32.990494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b47c2-9f00-4776-91b2-3d5f4d9af541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:32.991143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b47c2-9f00-4776-91b2-3d5f4d9af541 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:32.991419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b47c2-9f00-4776-91b2-3d5f4d9af541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:32.991689 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0b47c2-9f00-4776-91b2-3d5f4d9af541 admin admin] PATH_INFO: `/v3/projects/efd6d0d6b8f14f2ea31e438570d0a6cf/users/66fa6728fc894b3da1fbf57ac91316b0/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:32.993636 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b47c2-9f00-4776-91b2-3d5f4d9af541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efd6d0d6b8f14f2ea31e438570d0a6cf, user_id=66fa6728fc894b3da1fbf57ac91316b0, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:32.996908 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0b47c2-9f00-4776-91b2-3d5f4d9af541 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:33.009167 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1193/2374] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:28:32 2022] PUT /identity/v3/projects/efd6d0d6b8f14f2ea31e438570d0a6cf/users/66fa6728fc894b3da1fbf57ac91316b0/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:33.021280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabca384-54b5-4114-80cc-8aa5de40a546 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:33.021820 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabca384-54b5-4114-80cc-8aa5de40a546 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:33.022091 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabca384-54b5-4114-80cc-8aa5de40a546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:33.050941 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-dabca384-54b5-4114-80cc-8aa5de40a546 None None] MFA Rules not processed for user `66fa6728fc894b3da1fbf57ac91316b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:33.052156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dabca384-54b5-4114-80cc-8aa5de40a546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:33.136164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1182/2375] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:33 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:33.148206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b47c2-9f00-4776-91b2-3d5f4d9af541 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:33.150172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2d0b47c2-9f00-4776-91b2-3d5f4d9af541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:33.215275 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0b47c2-9f00-4776-91b2-3d5f4d9af541 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:33.219017 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d54a1d-b3d7-4d82-b878-146c7599dc09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:33.219530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d54a1d-b3d7-4d82-b878-146c7599dc09 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:33.219835 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d54a1d-b3d7-4d82-b878-146c7599dc09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:33.220076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d54a1d-b3d7-4d82-b878-146c7599dc09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:33.283515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d54a1d-b3d7-4d82-b878-146c7599dc09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:33.285242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d54a1d-b3d7-4d82-b878-146c7599dc09 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:33.300805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1194/2376] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:33 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:39.123744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabca384-54b5-4114-80cc-8aa5de40a546 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:39.126628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dabca384-54b5-4114-80cc-8aa5de40a546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:39.139266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabca384-54b5-4114-80cc-8aa5de40a546 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:39.142964 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a2af0-b0bc-4b29-8556-f29621521dbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:39.143563 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a2af0-b0bc-4b29-8556-f29621521dbe service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:39.143842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a2af0-b0bc-4b29-8556-f29621521dbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:39.144117 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a2af0-b0bc-4b29-8556-f29621521dbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:39.182252 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:39.216823 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a2af0-b0bc-4b29-8556-f29621521dbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:39.219227 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a2af0-b0bc-4b29-8556-f29621521dbe service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:39.229307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1183/2377] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:39 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:40.084299 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d54a1d-b3d7-4d82-b878-146c7599dc09 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:40.096539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d54a1d-b3d7-4d82-b878-146c7599dc09 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:40.100405 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83f5c0-4152-4848-bfb9-98795b466c82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:40.101650 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83f5c0-4152-4848-bfb9-98795b466c82 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:40.102300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83f5c0-4152-4848-bfb9-98795b466c82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:40.103658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd83f5c0-4152-4848-bfb9-98795b466c82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:40.124598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83f5c0-4152-4848-bfb9-98795b466c82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:40.126065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd83f5c0-4152-4848-bfb9-98795b466c82 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:40.141424 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1195/2378] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:48.843257 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef254c8-c482-4f82-b29b-57da072975ce None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:48.843772 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef254c8-c482-4f82-b29b-57da072975ce None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:48.843772 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef254c8-c482-4f82-b29b-57da072975ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:48.866316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3ef254c8-c482-4f82-b29b-57da072975ce None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:48.867620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3ef254c8-c482-4f82-b29b-57da072975ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:48.930252 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1184/2379] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 15:28:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:48.975815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83f5c0-4152-4848-bfb9-98795b466c82 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:48.977314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bd83f5c0-4152-4848-bfb9-98795b466c82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:49.031452 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:49.051013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd83f5c0-4152-4848-bfb9-98795b466c82 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:49.056153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8ea07-e0c6-4d44-b1a2-f000f254a047 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:49.056763 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8ea07-e0c6-4d44-b1a2-f000f254a047 service swift] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:49.057047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8ea07-e0c6-4d44-b1a2-f000f254a047 service swift] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:49.057328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a8ea07-e0c6-4d44-b1a2-f000f254a047 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:49.066280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8ea07-e0c6-4d44-b1a2-f000f254a047 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:49.068220 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a8ea07-e0c6-4d44-b1a2-f000f254a047 service swift] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:49.077850 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1196/2380] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:28:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:28:49.604423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623c0e3-37c2-4d5e-8fbe-46a893b4ef87 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:49.605228 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623c0e3-37c2-4d5e-8fbe-46a893b4ef87 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:49.605669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623c0e3-37c2-4d5e-8fbe-46a893b4ef87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:49.629604 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6623c0e3-37c2-4d5e-8fbe-46a893b4ef87 None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:49.631003 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6623c0e3-37c2-4d5e-8fbe-46a893b4ef87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:49.641419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1185/2381] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 24 15:28:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:49.660190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8ea07-e0c6-4d44-b1a2-f000f254a047 service swift] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:49.672380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a8ea07-e0c6-4d44-b1a2-f000f254a047 service swift] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:49.677477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87981350-b2d8-40e6-8622-45cf1bb56620 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:49.678299 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87981350-b2d8-40e6-8622-45cf1bb56620 service swift] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:49.678678 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87981350-b2d8-40e6-8622-45cf1bb56620 service swift] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:49.679065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87981350-b2d8-40e6-8622-45cf1bb56620 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:49.688891 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87981350-b2d8-40e6-8622-45cf1bb56620 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:49.690697 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87981350-b2d8-40e6-8622-45cf1bb56620 service swift] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:49.699703 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1197/2382] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:28:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:28:49.969658 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225cda3-476f-402b-9313-1da7527fcd61 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:49.970173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225cda3-476f-402b-9313-1da7527fcd61 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:49.970173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6225cda3-476f-402b-9313-1da7527fcd61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:49.990109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6225cda3-476f-402b-9313-1da7527fcd61 None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:49.991363 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6225cda3-476f-402b-9313-1da7527fcd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:49.999120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1186/2383] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 15:28:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:50.013901 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87981350-b2d8-40e6-8622-45cf1bb56620 service swift] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:50.025493 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87981350-b2d8-40e6-8622-45cf1bb56620 service swift] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:50.031092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e9787-e17d-4a9c-af30-632fdd01f81c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:50.031666 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e9787-e17d-4a9c-af30-632fdd01f81c service swift] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:50.031949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e9787-e17d-4a9c-af30-632fdd01f81c service swift] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:50.032252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e9787-e17d-4a9c-af30-632fdd01f81c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:50.052984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e9787-e17d-4a9c-af30-632fdd01f81c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:50.054600 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e9787-e17d-4a9c-af30-632fdd01f81c service swift] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:50.069505 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1198/2384] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:28:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:28:50.396516 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d587e9c-cce2-4316-b814-14801a4db1d2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:50.396801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d587e9c-cce2-4316-b814-14801a4db1d2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:50.397071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d587e9c-cce2-4316-b814-14801a4db1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:50.415898 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0d587e9c-cce2-4316-b814-14801a4db1d2 None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:50.417125 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0d587e9c-cce2-4316-b814-14801a4db1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:50.424827 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1187/2385] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 24 15:28:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:50.437639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e9787-e17d-4a9c-af30-632fdd01f81c service swift] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:50.448290 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e9787-e17d-4a9c-af30-632fdd01f81c service swift] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:50.453550 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34072ec9-4af0-4670-a4f0-6947e06e3706 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:50.454139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34072ec9-4af0-4670-a4f0-6947e06e3706 service swift] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:50.454456 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34072ec9-4af0-4670-a4f0-6947e06e3706 service swift] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:50.454728 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34072ec9-4af0-4670-a4f0-6947e06e3706 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:50.463016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34072ec9-4af0-4670-a4f0-6947e06e3706 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:50.464590 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34072ec9-4af0-4670-a4f0-6947e06e3706 service swift] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:50.472572 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1199/2386] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:28:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:28:54.732438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d587e9c-cce2-4316-b814-14801a4db1d2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:54.733919 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0d587e9c-cce2-4316-b814-14801a4db1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:54.809705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d587e9c-cce2-4316-b814-14801a4db1d2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:54.815751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb91a2e-858b-4362-a608-b4256b53cac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:54.816437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb91a2e-858b-4362-a608-b4256b53cac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:54.816666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb91a2e-858b-4362-a608-b4256b53cac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:54.816938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb91a2e-858b-4362-a608-b4256b53cac9 admin admin] PATH_INFO: `/v3/users/23c2101de3874d9ab296aa1e6b5e9b2c` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:54.818077 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb91a2e-858b-4362-a608-b4256b53cac9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c2101de3874d9ab296aa1e6b5e9b2c)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:54.820110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb91a2e-858b-4362-a608-b4256b53cac9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:54.866007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4cb91a2e-858b-4362-a608-b4256b53cac9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c2101de3874d9ab296aa1e6b5e9b2c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:54.866713 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4cb91a2e-858b-4362-a608-b4256b53cac9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c2101de3874d9ab296aa1e6b5e9b2c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:54.875285 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:54.881944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4cb91a2e-858b-4362-a608-b4256b53cac9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c2101de3874d9ab296aa1e6b5e9b2c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:54.890983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4cb91a2e-858b-4362-a608-b4256b53cac9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c2101de3874d9ab296aa1e6b5e9b2c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:54.891596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4cb91a2e-858b-4362-a608-b4256b53cac9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c2101de3874d9ab296aa1e6b5e9b2c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:54.908014 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1188/2387] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:28:54 2022] DELETE /identity/v3/users/23c2101de3874d9ab296aa1e6b5e9b2c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:54.910747 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91207b7a-e5ce-47b3-9fe6-aa5f29fe5dde None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:54.910747 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91207b7a-e5ce-47b3-9fe6-aa5f29fe5dde None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:54.913535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91207b7a-e5ce-47b3-9fe6-aa5f29fe5dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:54.934938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-91207b7a-e5ce-47b3-9fe6-aa5f29fe5dde None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:54.936253 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-91207b7a-e5ce-47b3-9fe6-aa5f29fe5dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:54.972341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb91a2e-858b-4362-a608-b4256b53cac9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:54.973838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4cb91a2e-858b-4362-a608-b4256b53cac9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:55.011908 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1200/2388] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:28:55.023531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91207b7a-e5ce-47b3-9fe6-aa5f29fe5dde None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:55.024957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-91207b7a-e5ce-47b3-9fe6-aa5f29fe5dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:55.072692 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb91a2e-858b-4362-a608-b4256b53cac9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:55.077759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c8056-0387-46ea-bae6-a2939d9b46ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:55.078429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c8056-0387-46ea-bae6-a2939d9b46ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:55.078708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c8056-0387-46ea-bae6-a2939d9b46ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:55.078977 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c8056-0387-46ea-bae6-a2939d9b46ab admin admin] PATH_INFO: `/v3/users/55e7550911424451be8b3bb966f66bd9` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:55.080658 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c8056-0387-46ea-bae6-a2939d9b46ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=55e7550911424451be8b3bb966f66bd9)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:55.082633 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8c8056-0387-46ea-bae6-a2939d9b46ab admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:55.136885 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91207b7a-e5ce-47b3-9fe6-aa5f29fe5dde None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:55.142310 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64793fb-889d-4b35-905c-63cb2e188429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:55.142984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64793fb-889d-4b35-905c-63cb2e188429 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:55.143286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64793fb-889d-4b35-905c-63cb2e188429 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:55.143706 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64793fb-889d-4b35-905c-63cb2e188429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:55.158764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64793fb-889d-4b35-905c-63cb2e188429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:55.160521 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64793fb-889d-4b35-905c-63cb2e188429 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:55.179946 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1201/2389] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:28:55.195136 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1f8c8056-0387-46ea-bae6-a2939d9b46ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55e7550911424451be8b3bb966f66bd9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:55.195136 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1f8c8056-0387-46ea-bae6-a2939d9b46ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55e7550911424451be8b3bb966f66bd9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:55.236626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1f8c8056-0387-46ea-bae6-a2939d9b46ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55e7550911424451be8b3bb966f66bd9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:55.253767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1f8c8056-0387-46ea-bae6-a2939d9b46ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55e7550911424451be8b3bb966f66bd9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:55.255589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-1f8c8056-0387-46ea-bae6-a2939d9b46ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55e7550911424451be8b3bb966f66bd9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:55.277188 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1189/2390] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:28:54 2022] DELETE /identity/v3/users/55e7550911424451be8b3bb966f66bd9 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:28:55.283505 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333a9a2-f49c-4a44-969e-69446c50471a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:55.283788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333a9a2-f49c-4a44-969e-69446c50471a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:55.284061 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d333a9a2-f49c-4a44-969e-69446c50471a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:55.321273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d333a9a2-f49c-4a44-969e-69446c50471a None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:55.322567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d333a9a2-f49c-4a44-969e-69446c50471a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:55.376504 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:55.401195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c8056-0387-46ea-bae6-a2939d9b46ab admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:55.403310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1f8c8056-0387-46ea-bae6-a2939d9b46ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:55.416446 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1202/2391] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:55.440101 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333a9a2-f49c-4a44-969e-69446c50471a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:55.441671 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d333a9a2-f49c-4a44-969e-69446c50471a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:55.485662 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:55.492416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8c8056-0387-46ea-bae6-a2939d9b46ab admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:55.499264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:55.500087 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:55.500367 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:55.500845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16 admin admin] PATH_INFO: `/v3/projects/f63eb112769442ddbbe718fd4169cac7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:55.502277 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f63eb112769442ddbbe718fd4169cac7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:55.505085 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:55.513689 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d333a9a2-f49c-4a44-969e-69446c50471a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:55.518781 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc548793-d429-43f0-b6f9-9aabea3bf979 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:55.519533 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc548793-d429-43f0-b6f9-9aabea3bf979 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:55.519928 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc548793-d429-43f0-b6f9-9aabea3bf979 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:55.520349 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc548793-d429-43f0-b6f9-9aabea3bf979 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:55.534566 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc548793-d429-43f0-b6f9-9aabea3bf979 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:55.536058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc548793-d429-43f0-b6f9-9aabea3bf979 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:55.547773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1203/2392] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:55.564760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f63eb112769442ddbbe718fd4169cac7', 'request_id': 'req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:55.570585 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f63eb112769442ddbbe718fd4169cac7', 'request_id': 'req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:55.579726 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f63eb112769442ddbbe718fd4169cac7', 'request_id': 'req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:55.597610 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16 admin admin] The token cache is being invalidate because project f63eb112769442ddbbe718fd4169cac7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:28:55.597938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:55.598596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:55.600138 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1190/2393] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:28:55 2022] DELETE /identity/v3/projects/f63eb112769442ddbbe718fd4169cac7 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:28:55.867331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc548793-d429-43f0-b6f9-9aabea3bf979 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:55.868910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bc548793-d429-43f0-b6f9-9aabea3bf979 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:55.882663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc548793-d429-43f0-b6f9-9aabea3bf979 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:55.943816 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482d8ef-08d0-4123-8620-98908146f524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:55.944516 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482d8ef-08d0-4123-8620-98908146f524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:55.944706 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482d8ef-08d0-4123-8620-98908146f524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:55.944978 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482d8ef-08d0-4123-8620-98908146f524 admin admin] PATH_INFO: `/v3/projects/cba4ef8af239471b9bea1535e6e272af` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:55.946106 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482d8ef-08d0-4123-8620-98908146f524 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba4ef8af239471b9bea1535e6e272af)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:55.948043 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2482d8ef-08d0-4123-8620-98908146f524 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:55.999187 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2482d8ef-08d0-4123-8620-98908146f524 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba4ef8af239471b9bea1535e6e272af', 'request_id': 'req-2482d8ef-08d0-4123-8620-98908146f524'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.005723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2482d8ef-08d0-4123-8620-98908146f524 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba4ef8af239471b9bea1535e6e272af', 'request_id': 'req-2482d8ef-08d0-4123-8620-98908146f524'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.015151 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2482d8ef-08d0-4123-8620-98908146f524 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba4ef8af239471b9bea1535e6e272af', 'request_id': 'req-2482d8ef-08d0-4123-8620-98908146f524'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.026131 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2482d8ef-08d0-4123-8620-98908146f524 admin admin] The token cache is being invalidate because project cba4ef8af239471b9bea1535e6e272af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:28:56.026506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2482d8ef-08d0-4123-8620-98908146f524 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.027145 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2482d8ef-08d0-4123-8620-98908146f524 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.028626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1204/2394] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:28:55 2022] DELETE /identity/v3/projects/cba4ef8af239471b9bea1535e6e272af => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:56.034288 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:56.035396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:56.048480 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a7690e-f2b0-491a-b99c-99aa1cbfba16 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:56.113281 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92531c-e8f0-4dbb-aa9b-12ee70a91ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:56.113924 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92531c-e8f0-4dbb-aa9b-12ee70a91ee3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:56.114202 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92531c-e8f0-4dbb-aa9b-12ee70a91ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:56.114503 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92531c-e8f0-4dbb-aa9b-12ee70a91ee3 admin admin] PATH_INFO: `/v3/users/2081d751b86a404ba89b4c41c00f386a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:56.115650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92531c-e8f0-4dbb-aa9b-12ee70a91ee3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2081d751b86a404ba89b4c41c00f386a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:56.117569 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92531c-e8f0-4dbb-aa9b-12ee70a91ee3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:56.165644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5a92531c-e8f0-4dbb-aa9b-12ee70a91ee3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2081d751b86a404ba89b4c41c00f386a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.166323 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5a92531c-e8f0-4dbb-aa9b-12ee70a91ee3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2081d751b86a404ba89b4c41c00f386a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.178588 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5a92531c-e8f0-4dbb-aa9b-12ee70a91ee3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2081d751b86a404ba89b4c41c00f386a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.187857 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5a92531c-e8f0-4dbb-aa9b-12ee70a91ee3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2081d751b86a404ba89b4c41c00f386a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.188470 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5a92531c-e8f0-4dbb-aa9b-12ee70a91ee3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2081d751b86a404ba89b4c41c00f386a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.202302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1191/2395] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:28:56 2022] DELETE /identity/v3/users/2081d751b86a404ba89b4c41c00f386a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:28:56.508178 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482d8ef-08d0-4123-8620-98908146f524 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:56.510059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2482d8ef-08d0-4123-8620-98908146f524 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:56.541080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482d8ef-08d0-4123-8620-98908146f524 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:56.557067 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:56.614413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fad629-737b-4c21-9f42-b330edf2bd21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:56.615057 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fad629-737b-4c21-9f42-b330edf2bd21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:56.615335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fad629-737b-4c21-9f42-b330edf2bd21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:56.615621 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fad629-737b-4c21-9f42-b330edf2bd21 admin admin] PATH_INFO: `/v3/projects/d99ccdbf77564700b1c51456dc1d677d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:56.616769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fad629-737b-4c21-9f42-b330edf2bd21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d99ccdbf77564700b1c51456dc1d677d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:56.618778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fad629-737b-4c21-9f42-b330edf2bd21 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:56.669042 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-26fad629-737b-4c21-9f42-b330edf2bd21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd99ccdbf77564700b1c51456dc1d677d', 'request_id': 'req-26fad629-737b-4c21-9f42-b330edf2bd21'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.678517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-26fad629-737b-4c21-9f42-b330edf2bd21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd99ccdbf77564700b1c51456dc1d677d', 'request_id': 'req-26fad629-737b-4c21-9f42-b330edf2bd21'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.688082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-26fad629-737b-4c21-9f42-b330edf2bd21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd99ccdbf77564700b1c51456dc1d677d', 'request_id': 'req-26fad629-737b-4c21-9f42-b330edf2bd21'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.699344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-26fad629-737b-4c21-9f42-b330edf2bd21 admin admin] The token cache is being invalidate because project d99ccdbf77564700b1c51456dc1d677d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:28:56.699632 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-26fad629-737b-4c21-9f42-b330edf2bd21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.700279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-26fad629-737b-4c21-9f42-b330edf2bd21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.701778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1205/2396] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:28:56 2022] DELETE /identity/v3/projects/d99ccdbf77564700b1c51456dc1d677d => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:56.705910 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92531c-e8f0-4dbb-aa9b-12ee70a91ee3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:56.707415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5a92531c-e8f0-4dbb-aa9b-12ee70a91ee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:56.716677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:56.720756 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92531c-e8f0-4dbb-aa9b-12ee70a91ee3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:56.782956 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628a3b48-8530-4748-9a7d-eea287fb60ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:56.783636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a3b48-8530-4748-9a7d-eea287fb60ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:56.783919 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a3b48-8530-4748-9a7d-eea287fb60ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:56.784194 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628a3b48-8530-4748-9a7d-eea287fb60ab admin admin] PATH_INFO: `/v3/users/66fa6728fc894b3da1fbf57ac91316b0` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:56.785333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a3b48-8530-4748-9a7d-eea287fb60ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=66fa6728fc894b3da1fbf57ac91316b0)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:56.787614 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628a3b48-8530-4748-9a7d-eea287fb60ab admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:56.846392 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-628a3b48-8530-4748-9a7d-eea287fb60ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66fa6728fc894b3da1fbf57ac91316b0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.847281 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-628a3b48-8530-4748-9a7d-eea287fb60ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66fa6728fc894b3da1fbf57ac91316b0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.861701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-628a3b48-8530-4748-9a7d-eea287fb60ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66fa6728fc894b3da1fbf57ac91316b0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.869218 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-628a3b48-8530-4748-9a7d-eea287fb60ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66fa6728fc894b3da1fbf57ac91316b0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.869988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-628a3b48-8530-4748-9a7d-eea287fb60ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66fa6728fc894b3da1fbf57ac91316b0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:56.884983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1192/2397] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:28:56 2022] DELETE /identity/v3/users/66fa6728fc894b3da1fbf57ac91316b0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:57.204760 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fad629-737b-4c21-9f42-b330edf2bd21 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:57.206345 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-26fad629-737b-4c21-9f42-b330edf2bd21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:57.220017 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fad629-737b-4c21-9f42-b330edf2bd21 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:57.288413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b42edf6-08cb-4540-98f2-c005ca121ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:57.289473 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42edf6-08cb-4540-98f2-c005ca121ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:57.290275 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42edf6-08cb-4540-98f2-c005ca121ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:57.290581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b42edf6-08cb-4540-98f2-c005ca121ba2 admin admin] PATH_INFO: `/v3/projects/efd6d0d6b8f14f2ea31e438570d0a6cf` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:57.292699 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42edf6-08cb-4540-98f2-c005ca121ba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=efd6d0d6b8f14f2ea31e438570d0a6cf)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:57.295852 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b42edf6-08cb-4540-98f2-c005ca121ba2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:57.347298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2b42edf6-08cb-4540-98f2-c005ca121ba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efd6d0d6b8f14f2ea31e438570d0a6cf', 'request_id': 'req-2b42edf6-08cb-4540-98f2-c005ca121ba2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:57.348436 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:57.357822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2b42edf6-08cb-4540-98f2-c005ca121ba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efd6d0d6b8f14f2ea31e438570d0a6cf', 'request_id': 'req-2b42edf6-08cb-4540-98f2-c005ca121ba2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:57.368590 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2b42edf6-08cb-4540-98f2-c005ca121ba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efd6d0d6b8f14f2ea31e438570d0a6cf', 'request_id': 'req-2b42edf6-08cb-4540-98f2-c005ca121ba2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:57.380168 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2b42edf6-08cb-4540-98f2-c005ca121ba2 admin admin] The token cache is being invalidate because project efd6d0d6b8f14f2ea31e438570d0a6cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:28:57.380454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2b42edf6-08cb-4540-98f2-c005ca121ba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:57.381100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2b42edf6-08cb-4540-98f2-c005ca121ba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:57.382624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1206/2398] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:28:57 2022] DELETE /identity/v3/projects/efd6d0d6b8f14f2ea31e438570d0a6cf => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:57.387982 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0efc83-9162-4697-bb7d-2192eb570fc7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:57.388429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0efc83-9162-4697-bb7d-2192eb570fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:57.388522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0efc83-9162-4697-bb7d-2192eb570fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:57.409719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6d0efc83-9162-4697-bb7d-2192eb570fc7 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:57.411339 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6d0efc83-9162-4697-bb7d-2192eb570fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:57.485219 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1193/2399] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:57.493375 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4af0f-862c-4d99-930a-99fcc2e335b9 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:57.494082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4af0f-862c-4d99-930a-99fcc2e335b9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:57.494082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf4af0f-862c-4d99-930a-99fcc2e335b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:57.518061 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-1bf4af0f-862c-4d99-930a-99fcc2e335b9 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:57.518655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1bf4af0f-862c-4d99-930a-99fcc2e335b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:57.527282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1207/2400] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:28:57.532044 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0efc83-9162-4697-bb7d-2192eb570fc7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:57.550881 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0efc83-9162-4697-bb7d-2192eb570fc7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:57.559556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeca4c6-257b-4aa9-8959-8e05de0f8b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:57.559556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeca4c6-257b-4aa9-8959-8e05de0f8b5a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:57.559556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeca4c6-257b-4aa9-8959-8e05de0f8b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:57.559556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeca4c6-257b-4aa9-8959-8e05de0f8b5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:57.559944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeca4c6-257b-4aa9-8959-8e05de0f8b5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:57.560030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeca4c6-257b-4aa9-8959-8e05de0f8b5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:57.561614 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceeca4c6-257b-4aa9-8959-8e05de0f8b5a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:57.575894 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1194/2401] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:28:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:28:57.580242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4af0f-862c-4d99-930a-99fcc2e335b9 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:57.590696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf4af0f-862c-4d99-930a-99fcc2e335b9 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:57.595910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ab404-4582-4241-a1bf-d8914c50a047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:57.596517 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ab404-4582-4241-a1bf-d8914c50a047 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:57.596809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ab404-4582-4241-a1bf-d8914c50a047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:57.597160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7ab404-4582-4241-a1bf-d8914c50a047 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:57.597797 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ab404-4582-4241-a1bf-d8914c50a047 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:57.599730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7ab404-4582-4241-a1bf-d8914c50a047 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:57.628435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1208/2402] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:28:57 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:57.635428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeca4c6-257b-4aa9-8959-8e05de0f8b5a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:57.647237 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeca4c6-257b-4aa9-8959-8e05de0f8b5a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:57.657216 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:57.718817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5616d-db23-400a-83ab-82771f479720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:57.719431 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5616d-db23-400a-83ab-82771f479720 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:57.719710 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5616d-db23-400a-83ab-82771f479720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:57.719981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5616d-db23-400a-83ab-82771f479720 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:57.720835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5616d-db23-400a-83ab-82771f479720 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:57.723396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5616d-db23-400a-83ab-82771f479720 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:57.756124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1195/2403] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:28:57 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:28:57.760476 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ab404-4582-4241-a1bf-d8914c50a047 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:57.774447 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7ab404-4582-4241-a1bf-d8914c50a047 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:57.780103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8395b-ef99-4d68-bdf3-b8f3fae8f57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:57.781013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8395b-ef99-4d68-bdf3-b8f3fae8f57e admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:57.781935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8395b-ef99-4d68-bdf3-b8f3fae8f57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:57.782317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8395b-ef99-4d68-bdf3-b8f3fae8f57e admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:57.783168 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8395b-ef99-4d68-bdf3-b8f3fae8f57e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:57.784841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8395b-ef99-4d68-bdf3-b8f3fae8f57e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:57.796597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1209/2404] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:28:57.801664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5616d-db23-400a-83ab-82771f479720 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:57.814433 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5616d-db23-400a-83ab-82771f479720 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:57.819191 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05d520a-3109-4770-805c-f00042a7fed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:57.819791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d520a-3109-4770-805c-f00042a7fed8 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:57.820069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d520a-3109-4770-805c-f00042a7fed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:57.820340 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05d520a-3109-4770-805c-f00042a7fed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:57.820979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d520a-3109-4770-805c-f00042a7fed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:57.822600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05d520a-3109-4770-805c-f00042a7fed8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:57.836909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1196/2405] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:28:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:28:57.841432 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8395b-ef99-4d68-bdf3-b8f3fae8f57e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:57.851839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8395b-ef99-4d68-bdf3-b8f3fae8f57e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:57.856608 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6712238-20a6-4b2f-9eb4-4702d91a80f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:57.857314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6712238-20a6-4b2f-9eb4-4702d91a80f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:57.857593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6712238-20a6-4b2f-9eb4-4702d91a80f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:57.857863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6712238-20a6-4b2f-9eb4-4702d91a80f0 admin admin] PATH_INFO: `/v3/projects/d1610a6082d143d1b0916094775078ee/users/e7716a388b6c4b70ac59668e803d7201/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:57.873516 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6712238-20a6-4b2f-9eb4-4702d91a80f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1610a6082d143d1b0916094775078ee, user_id=e7716a388b6c4b70ac59668e803d7201, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:57.876273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6712238-20a6-4b2f-9eb4-4702d91a80f0 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:57.891620 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1210/2406] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:28:57 2022] PUT /identity/v3/projects/d1610a6082d143d1b0916094775078ee/users/e7716a388b6c4b70ac59668e803d7201/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:57.906258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5256a59-9b62-489d-90ee-9dafa10e3fd0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:57.906519 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5256a59-9b62-489d-90ee-9dafa10e3fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:57.906788 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5256a59-9b62-489d-90ee-9dafa10e3fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:57.939505 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e5256a59-9b62-489d-90ee-9dafa10e3fd0 None None] MFA Rules not processed for user `e7716a388b6c4b70ac59668e803d7201`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:57.940818 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e5256a59-9b62-489d-90ee-9dafa10e3fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:58.019603 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1197/2407] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:58.031275 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6712238-20a6-4b2f-9eb4-4702d91a80f0 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:58.033212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a6712238-20a6-4b2f-9eb4-4702d91a80f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:58.094446 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6712238-20a6-4b2f-9eb4-4702d91a80f0 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:58.099702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fd0759-956a-41c1-ac6c-41165562d3c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:58.100321 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd0759-956a-41c1-ac6c-41165562d3c4 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:58.100598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd0759-956a-41c1-ac6c-41165562d3c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:58.100866 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fd0759-956a-41c1-ac6c-41165562d3c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:58.110481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd0759-956a-41c1-ac6c-41165562d3c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:58.111985 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fd0759-956a-41c1-ac6c-41165562d3c4 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:58.121738 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1211/2408] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:28:58.912312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5256a59-9b62-489d-90ee-9dafa10e3fd0 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:58.913791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e5256a59-9b62-489d-90ee-9dafa10e3fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:58.982175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5256a59-9b62-489d-90ee-9dafa10e3fd0 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:58.987302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a73525-6333-477d-96ee-0ab707afee95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:58.987932 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a73525-6333-477d-96ee-0ab707afee95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:58.988206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a73525-6333-477d-96ee-0ab707afee95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:58.988473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a73525-6333-477d-96ee-0ab707afee95 admin admin] PATH_INFO: `/v3/users/43880a5b93a14a38907c232a7ce22407` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:58.989897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a73525-6333-477d-96ee-0ab707afee95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43880a5b93a14a38907c232a7ce22407)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:58.991760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a73525-6333-477d-96ee-0ab707afee95 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:59.014863 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:59.041429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-09a73525-6333-477d-96ee-0ab707afee95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43880a5b93a14a38907c232a7ce22407'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:59.042108 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-09a73525-6333-477d-96ee-0ab707afee95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43880a5b93a14a38907c232a7ce22407'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:59.054155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-09a73525-6333-477d-96ee-0ab707afee95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43880a5b93a14a38907c232a7ce22407'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:59.062750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-09a73525-6333-477d-96ee-0ab707afee95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43880a5b93a14a38907c232a7ce22407'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:59.063421 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-09a73525-6333-477d-96ee-0ab707afee95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43880a5b93a14a38907c232a7ce22407'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:59.080241 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1198/2409] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:28:58 2022] DELETE /identity/v3/users/43880a5b93a14a38907c232a7ce22407 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:59.083197 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4aeb61-2236-4d7b-ae2f-508333ec5b0a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:59.083640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4aeb61-2236-4d7b-ae2f-508333ec5b0a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:59.083765 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4aeb61-2236-4d7b-ae2f-508333ec5b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:59.106343 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-2a4aeb61-2236-4d7b-ae2f-508333ec5b0a None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:59.107581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2a4aeb61-2236-4d7b-ae2f-508333ec5b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:59.169777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1212/2410] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:59.179578 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a73525-6333-477d-96ee-0ab707afee95 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:59.181000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-09a73525-6333-477d-96ee-0ab707afee95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:59.242785 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a73525-6333-477d-96ee-0ab707afee95 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:59.246837 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e6c3a-1edd-444d-a94c-dcf31f61b4e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:59.247445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e6c3a-1edd-444d-a94c-dcf31f61b4e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:59.247725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e6c3a-1edd-444d-a94c-dcf31f61b4e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:59.247999 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e6c3a-1edd-444d-a94c-dcf31f61b4e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:59.257230 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e6c3a-1edd-444d-a94c-dcf31f61b4e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:59.258692 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e6c3a-1edd-444d-a94c-dcf31f61b4e8 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:59.268743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1199/2411] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:28:59.645958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4aeb61-2236-4d7b-ae2f-508333ec5b0a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:59.647646 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2a4aeb61-2236-4d7b-ae2f-508333ec5b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:59.661202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4aeb61-2236-4d7b-ae2f-508333ec5b0a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:59.665989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5db945-75b0-4ebc-bc6a-24d453286af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:59.666657 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5db945-75b0-4ebc-bc6a-24d453286af4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:59.667566 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5db945-75b0-4ebc-bc6a-24d453286af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:59.667883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5db945-75b0-4ebc-bc6a-24d453286af4 admin admin] PATH_INFO: `/v3/projects/d3d7a835b9994a7e833a40ee6208f0e9` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:59.669288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5db945-75b0-4ebc-bc6a-24d453286af4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3d7a835b9994a7e833a40ee6208f0e9)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:59.671236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5db945-75b0-4ebc-bc6a-24d453286af4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:59.683474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:28:59.724900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0c5db945-75b0-4ebc-bc6a-24d453286af4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3d7a835b9994a7e833a40ee6208f0e9', 'request_id': 'req-0c5db945-75b0-4ebc-bc6a-24d453286af4'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:59.732051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0c5db945-75b0-4ebc-bc6a-24d453286af4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3d7a835b9994a7e833a40ee6208f0e9', 'request_id': 'req-0c5db945-75b0-4ebc-bc6a-24d453286af4'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:59.740552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0c5db945-75b0-4ebc-bc6a-24d453286af4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3d7a835b9994a7e833a40ee6208f0e9', 'request_id': 'req-0c5db945-75b0-4ebc-bc6a-24d453286af4'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:59.754109 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0c5db945-75b0-4ebc-bc6a-24d453286af4 admin admin] The token cache is being invalidate because project d3d7a835b9994a7e833a40ee6208f0e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:28:59.754109 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0c5db945-75b0-4ebc-bc6a-24d453286af4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:59.755053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0c5db945-75b0-4ebc-bc6a-24d453286af4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:28:59.757450 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1213/2412] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:28:59 2022] DELETE /identity/v3/projects/d3d7a835b9994a7e833a40ee6208f0e9 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:28:59.763023 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc23bc-c1fc-4ce9-a21d-6bee9e2b6844 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:59.763311 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc23bc-c1fc-4ce9-a21d-6bee9e2b6844 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:59.763715 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc23bc-c1fc-4ce9-a21d-6bee9e2b6844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:59.788334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-bbbc23bc-c1fc-4ce9-a21d-6bee9e2b6844 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:59.789881 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bbbc23bc-c1fc-4ce9-a21d-6bee9e2b6844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:59.866381 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1200/2413] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:59.875500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3509ab-f2f5-41ef-86c0-68decfd3c305 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:59.875840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3509ab-f2f5-41ef-86c0-68decfd3c305 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:59.876051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3509ab-f2f5-41ef-86c0-68decfd3c305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:59.896974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9a3509ab-f2f5-41ef-86c0-68decfd3c305 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:28:59.898600 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9a3509ab-f2f5-41ef-86c0-68decfd3c305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:28:59.908735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1214/2414] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:28:59.912560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc23bc-c1fc-4ce9-a21d-6bee9e2b6844 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:59.923126 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc23bc-c1fc-4ce9-a21d-6bee9e2b6844 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:59.927767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a34d1-e495-41b9-ae07-39618a51304c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:59.928368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a34d1-e495-41b9-ae07-39618a51304c admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:59.928641 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a34d1-e495-41b9-ae07-39618a51304c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:59.928907 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6a34d1-e495-41b9-ae07-39618a51304c admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:59.929824 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a34d1-e495-41b9-ae07-39618a51304c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:28:59.930196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a34d1-e495-41b9-ae07-39618a51304c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:59.931823 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6a34d1-e495-41b9-ae07-39618a51304c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:28:59.945163 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1201/2415] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:28:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:28:59.949901 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3509ab-f2f5-41ef-86c0-68decfd3c305 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:28:59.960574 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3509ab-f2f5-41ef-86c0-68decfd3c305 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:28:59.966731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80400dda-9134-4da9-a405-a40c98ff97fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:28:59.967402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80400dda-9134-4da9-a405-a40c98ff97fa admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:28:59.967687 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80400dda-9134-4da9-a405-a40c98ff97fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:28:59.967960 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80400dda-9134-4da9-a405-a40c98ff97fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:28:59.968590 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80400dda-9134-4da9-a405-a40c98ff97fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:28:59.970982 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80400dda-9134-4da9-a405-a40c98ff97fa admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:00.006007 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1215/2416] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:28:59 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:29:00.008173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a34d1-e495-41b9-ae07-39618a51304c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:00.020536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6a34d1-e495-41b9-ae07-39618a51304c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:00.047204 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:00.082226 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e763b-d981-47f2-9cf9-01e580b491b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:00.082909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e763b-d981-47f2-9cf9-01e580b491b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:00.083195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e763b-d981-47f2-9cf9-01e580b491b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:00.083466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e763b-d981-47f2-9cf9-01e580b491b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:00.084100 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e763b-d981-47f2-9cf9-01e580b491b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:00.086091 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28e763b-d981-47f2-9cf9-01e580b491b0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:00.109042 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1202/2417] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:29:00 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:29:00.114192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80400dda-9134-4da9-a405-a40c98ff97fa admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:00.126143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80400dda-9134-4da9-a405-a40c98ff97fa admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:00.131102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5cc04-a85d-49fa-afc1-dcdd5460e0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:00.131844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5cc04-a85d-49fa-afc1-dcdd5460e0df admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:00.132123 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5cc04-a85d-49fa-afc1-dcdd5460e0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:00.132398 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e5cc04-a85d-49fa-afc1-dcdd5460e0df admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:00.133044 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5cc04-a85d-49fa-afc1-dcdd5460e0df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:00.134745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e5cc04-a85d-49fa-afc1-dcdd5460e0df admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:00.146129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1216/2418] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:29:00.150648 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e763b-d981-47f2-9cf9-01e580b491b0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:00.161633 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28e763b-d981-47f2-9cf9-01e580b491b0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:00.166465 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b4ed9-50ba-4839-abef-5a162e7f17cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:00.167099 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b4ed9-50ba-4839-abef-5a162e7f17cf admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:00.167381 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b4ed9-50ba-4839-abef-5a162e7f17cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:00.167652 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b4ed9-50ba-4839-abef-5a162e7f17cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:00.168294 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b4ed9-50ba-4839-abef-5a162e7f17cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:00.169877 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b4ed9-50ba-4839-abef-5a162e7f17cf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:00.180618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1203/2419] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:29:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:29:00.184852 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5cc04-a85d-49fa-afc1-dcdd5460e0df admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:00.195669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e5cc04-a85d-49fa-afc1-dcdd5460e0df admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:00.201848 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d157d37-002c-4eb7-be82-8600b6585eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:00.202534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d157d37-002c-4eb7-be82-8600b6585eb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:00.202811 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d157d37-002c-4eb7-be82-8600b6585eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:00.203083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d157d37-002c-4eb7-be82-8600b6585eb5 admin admin] PATH_INFO: `/v3/projects/780291a78ff54be5b744c2d5d9c6045b/users/ddd16557210b445fa066c1b1a43a8a8a/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:00.219570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d157d37-002c-4eb7-be82-8600b6585eb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=780291a78ff54be5b744c2d5d9c6045b, user_id=ddd16557210b445fa066c1b1a43a8a8a, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:00.222351 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d157d37-002c-4eb7-be82-8600b6585eb5 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:00.235838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1217/2420] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:29:00 2022] PUT /identity/v3/projects/780291a78ff54be5b744c2d5d9c6045b/users/ddd16557210b445fa066c1b1a43a8a8a/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:29:00.246116 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5ffbe-4573-4d10-a62b-f5d7230d0f7a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:00.246457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5ffbe-4573-4d10-a62b-f5d7230d0f7a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:00.246730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5ffbe-4573-4d10-a62b-f5d7230d0f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:00.275892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-05b5ffbe-4573-4d10-a62b-f5d7230d0f7a None None] MFA Rules not processed for user `ddd16557210b445fa066c1b1a43a8a8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:00.277587 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-05b5ffbe-4573-4d10-a62b-f5d7230d0f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:00.344381 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1204/2421] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:29:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:00.355931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d157d37-002c-4eb7-be82-8600b6585eb5 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:00.357684 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8d157d37-002c-4eb7-be82-8600b6585eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:00.416673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d157d37-002c-4eb7-be82-8600b6585eb5 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:00.421080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd1a0b6-5737-4217-9927-b2c5ba6cbf34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:00.421723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1a0b6-5737-4217-9927-b2c5ba6cbf34 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:00.422001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1a0b6-5737-4217-9927-b2c5ba6cbf34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:00.422312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd1a0b6-5737-4217-9927-b2c5ba6cbf34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:00.431898 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1a0b6-5737-4217-9927-b2c5ba6cbf34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:00.433499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd1a0b6-5737-4217-9927-b2c5ba6cbf34 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:00.442653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1218/2422] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:04.108906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5ffbe-4573-4d10-a62b-f5d7230d0f7a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:04.172165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5ffbe-4573-4d10-a62b-f5d7230d0f7a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:04.176696 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372bac1-1f3a-4cb5-bce4-167e099b92de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:04.177313 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372bac1-1f3a-4cb5-bce4-167e099b92de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:04.177588 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372bac1-1f3a-4cb5-bce4-167e099b92de admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:04.177861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a372bac1-1f3a-4cb5-bce4-167e099b92de admin admin] PATH_INFO: `/v3/users/ddd16557210b445fa066c1b1a43a8a8a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:04.178995 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372bac1-1f3a-4cb5-bce4-167e099b92de admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddd16557210b445fa066c1b1a43a8a8a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:04.180910 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a372bac1-1f3a-4cb5-bce4-167e099b92de admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:04.226390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:04.243633 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a372bac1-1f3a-4cb5-bce4-167e099b92de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddd16557210b445fa066c1b1a43a8a8a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:04.243633 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a372bac1-1f3a-4cb5-bce4-167e099b92de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddd16557210b445fa066c1b1a43a8a8a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:04.261356 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a372bac1-1f3a-4cb5-bce4-167e099b92de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddd16557210b445fa066c1b1a43a8a8a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:04.273160 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a372bac1-1f3a-4cb5-bce4-167e099b92de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddd16557210b445fa066c1b1a43a8a8a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:04.275365 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-a372bac1-1f3a-4cb5-bce4-167e099b92de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddd16557210b445fa066c1b1a43a8a8a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:04.309537 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1205/2423] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:29:04 2022] DELETE /identity/v3/users/ddd16557210b445fa066c1b1a43a8a8a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:29:04.313725 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f264846-d572-4fd2-b550-ac8cdf5e67ab None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:04.314202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f264846-d572-4fd2-b550-ac8cdf5e67ab None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:04.314508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f264846-d572-4fd2-b550-ac8cdf5e67ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:04.334970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-4f264846-d572-4fd2-b550-ac8cdf5e67ab None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:04.336496 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4f264846-d572-4fd2-b550-ac8cdf5e67ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:04.384509 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:04.439105 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1219/2424] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:04.449736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372bac1-1f3a-4cb5-bce4-167e099b92de admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:04.452055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a372bac1-1f3a-4cb5-bce4-167e099b92de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:04.534815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a372bac1-1f3a-4cb5-bce4-167e099b92de admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:04.539100 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0af23-71c1-4a04-a63e-b144275d85ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:04.540270 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0af23-71c1-4a04-a63e-b144275d85ba service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:04.540270 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0af23-71c1-4a04-a63e-b144275d85ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:04.540490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0af23-71c1-4a04-a63e-b144275d85ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:04.551326 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0af23-71c1-4a04-a63e-b144275d85ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:04.552924 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0af23-71c1-4a04-a63e-b144275d85ba service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:04.564079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1206/2425] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:04.878184 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f264846-d572-4fd2-b550-ac8cdf5e67ab None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:04.879883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4f264846-d572-4fd2-b550-ac8cdf5e67ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:04.895398 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f264846-d572-4fd2-b550-ac8cdf5e67ab None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:04.900159 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70af8da-076f-4798-99e6-2fd0b9f72dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:04.900788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70af8da-076f-4798-99e6-2fd0b9f72dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:04.901060 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70af8da-076f-4798-99e6-2fd0b9f72dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:04.901329 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70af8da-076f-4798-99e6-2fd0b9f72dc8 admin admin] PATH_INFO: `/v3/projects/780291a78ff54be5b744c2d5d9c6045b` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:04.902467 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70af8da-076f-4798-99e6-2fd0b9f72dc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=780291a78ff54be5b744c2d5d9c6045b)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:04.904479 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70af8da-076f-4798-99e6-2fd0b9f72dc8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:04.963966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c70af8da-076f-4798-99e6-2fd0b9f72dc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '780291a78ff54be5b744c2d5d9c6045b', 'request_id': 'req-c70af8da-076f-4798-99e6-2fd0b9f72dc8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:04.970770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c70af8da-076f-4798-99e6-2fd0b9f72dc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '780291a78ff54be5b744c2d5d9c6045b', 'request_id': 'req-c70af8da-076f-4798-99e6-2fd0b9f72dc8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:04.979104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c70af8da-076f-4798-99e6-2fd0b9f72dc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '780291a78ff54be5b744c2d5d9c6045b', 'request_id': 'req-c70af8da-076f-4798-99e6-2fd0b9f72dc8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:04.991710 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c70af8da-076f-4798-99e6-2fd0b9f72dc8 admin admin] The token cache is being invalidate because project 780291a78ff54be5b744c2d5d9c6045b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:29:04.992139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c70af8da-076f-4798-99e6-2fd0b9f72dc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:04.992879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c70af8da-076f-4798-99e6-2fd0b9f72dc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:04.994571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1220/2426] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:29:04 2022] DELETE /identity/v3/projects/780291a78ff54be5b744c2d5d9c6045b => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:29:05.000188 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c67ca-705c-4758-b223-b7f3ac84d2fa None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:05.000475 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c67ca-705c-4758-b223-b7f3ac84d2fa None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:05.000747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c67ca-705c-4758-b223-b7f3ac84d2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:05.023784 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e76c67ca-705c-4758-b223-b7f3ac84d2fa None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:05.025009 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e76c67ca-705c-4758-b223-b7f3ac84d2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:05.094110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1207/2427] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:29:05.102725 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f78a4e-b539-4ebc-ada6-9666ac248d51 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:05.103007 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f78a4e-b539-4ebc-ada6-9666ac248d51 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:05.103278 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f78a4e-b539-4ebc-ada6-9666ac248d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:05.126804 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-47f78a4e-b539-4ebc-ada6-9666ac248d51 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:05.127952 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-47f78a4e-b539-4ebc-ada6-9666ac248d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:05.136514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1221/2428] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:29:05.140770 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c67ca-705c-4758-b223-b7f3ac84d2fa None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:05.150670 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c67ca-705c-4758-b223-b7f3ac84d2fa None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:05.155374 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6b011-a97f-4961-9585-a38203db6705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:05.155973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6b011-a97f-4961-9585-a38203db6705 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:05.156251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6b011-a97f-4961-9585-a38203db6705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:05.156522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6b011-a97f-4961-9585-a38203db6705 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:05.157161 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6b011-a97f-4961-9585-a38203db6705 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:29:05.157479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6b011-a97f-4961-9585-a38203db6705 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:05.159375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6b011-a97f-4961-9585-a38203db6705 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:05.171058 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1208/2429] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:29:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:29:05.175633 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f78a4e-b539-4ebc-ada6-9666ac248d51 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:05.185591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f78a4e-b539-4ebc-ada6-9666ac248d51 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:05.190486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5c4bb-df35-4eb3-abc6-836974580f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:05.191123 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5c4bb-df35-4eb3-abc6-836974580f36 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:05.191403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5c4bb-df35-4eb3-abc6-836974580f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:05.191675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e5c4bb-df35-4eb3-abc6-836974580f36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:05.192298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5c4bb-df35-4eb3-abc6-836974580f36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:05.194242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e5c4bb-df35-4eb3-abc6-836974580f36 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:05.222874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1222/2430] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:29:05 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:29:05.227389 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6b011-a97f-4961-9585-a38203db6705 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:05.237864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6b011-a97f-4961-9585-a38203db6705 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:05.284979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:05.300574 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4324cf-f606-4726-9dcd-1f2f5a103725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:05.301265 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4324cf-f606-4726-9dcd-1f2f5a103725 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:05.301450 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4324cf-f606-4726-9dcd-1f2f5a103725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:05.301726 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4324cf-f606-4726-9dcd-1f2f5a103725 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:05.302393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4324cf-f606-4726-9dcd-1f2f5a103725 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:05.304429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4324cf-f606-4726-9dcd-1f2f5a103725 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:05.335411 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1209/2431] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:29:05 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:29:05.339954 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5c4bb-df35-4eb3-abc6-836974580f36 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:05.350184 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e5c4bb-df35-4eb3-abc6-836974580f36 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:05.359374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979e8c9-2138-4f5a-be4b-dfdcce480ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:05.360158 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979e8c9-2138-4f5a-be4b-dfdcce480ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:05.360575 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979e8c9-2138-4f5a-be4b-dfdcce480ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:05.360988 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f979e8c9-2138-4f5a-be4b-dfdcce480ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:05.361790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979e8c9-2138-4f5a-be4b-dfdcce480ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:05.363730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f979e8c9-2138-4f5a-be4b-dfdcce480ba3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:05.375267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1223/2432] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:29:05.379390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4324cf-f606-4726-9dcd-1f2f5a103725 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:05.392853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4324cf-f606-4726-9dcd-1f2f5a103725 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:05.397553 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50302b3-0d65-4705-aab5-9e1b97529321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:05.398155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50302b3-0d65-4705-aab5-9e1b97529321 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:05.398482 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50302b3-0d65-4705-aab5-9e1b97529321 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:05.398752 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50302b3-0d65-4705-aab5-9e1b97529321 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:05.399393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50302b3-0d65-4705-aab5-9e1b97529321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:05.400980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50302b3-0d65-4705-aab5-9e1b97529321 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:05.412151 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1210/2433] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:29:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:29:05.416565 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979e8c9-2138-4f5a-be4b-dfdcce480ba3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:05.427520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f979e8c9-2138-4f5a-be4b-dfdcce480ba3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:05.433028 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b468d4-7cec-4135-a12f-ad78bd253d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:05.433690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b468d4-7cec-4135-a12f-ad78bd253d17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:05.433966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b468d4-7cec-4135-a12f-ad78bd253d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:05.434255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b468d4-7cec-4135-a12f-ad78bd253d17 admin admin] PATH_INFO: `/v3/projects/d1651f8461a34e1ca016176e7dfb8469/users/5f828cb6c46a47628d3190c0f38f92b3/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:05.455024 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b468d4-7cec-4135-a12f-ad78bd253d17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1651f8461a34e1ca016176e7dfb8469, user_id=5f828cb6c46a47628d3190c0f38f92b3, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:05.457679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b468d4-7cec-4135-a12f-ad78bd253d17 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:05.471548 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1224/2434] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:29:05 2022] PUT /identity/v3/projects/d1651f8461a34e1ca016176e7dfb8469/users/5f828cb6c46a47628d3190c0f38f92b3/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:29:05.483691 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977f85e-9872-4d59-8d47-18ba4e229437 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:05.484131 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977f85e-9872-4d59-8d47-18ba4e229437 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:05.484551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977f85e-9872-4d59-8d47-18ba4e229437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:05.515142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-0977f85e-9872-4d59-8d47-18ba4e229437 None None] MFA Rules not processed for user `5f828cb6c46a47628d3190c0f38f92b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:05.516373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0977f85e-9872-4d59-8d47-18ba4e229437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:05.594147 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1211/2435] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:29:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:05.607955 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b468d4-7cec-4135-a12f-ad78bd253d17 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:05.609923 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f0b468d4-7cec-4135-a12f-ad78bd253d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:05.679242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b468d4-7cec-4135-a12f-ad78bd253d17 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:05.683362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef438560-513c-4ba1-b8a4-7769cf4092c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:05.683964 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef438560-513c-4ba1-b8a4-7769cf4092c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:05.684242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef438560-513c-4ba1-b8a4-7769cf4092c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:05.684513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef438560-513c-4ba1-b8a4-7769cf4092c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:05.694252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef438560-513c-4ba1-b8a4-7769cf4092c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:05.695751 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef438560-513c-4ba1-b8a4-7769cf4092c8 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:05.705826 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1225/2436] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:05 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:29:15.590820 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=45,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:15.701102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:21.504758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977f85e-9872-4d59-8d47-18ba4e229437 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:21.506433 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0977f85e-9872-4d59-8d47-18ba4e229437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:21.631901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0977f85e-9872-4d59-8d47-18ba4e229437 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:21.637425 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1687e03-291f-4858-a1c1-f66cfb18bc97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:21.638070 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1687e03-291f-4858-a1c1-f66cfb18bc97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:21.638382 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1687e03-291f-4858-a1c1-f66cfb18bc97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:21.638669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1687e03-291f-4858-a1c1-f66cfb18bc97 admin admin] PATH_INFO: `/v3/users/e7716a388b6c4b70ac59668e803d7201` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:21.639818 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1687e03-291f-4858-a1c1-f66cfb18bc97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7716a388b6c4b70ac59668e803d7201)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:21.641938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1687e03-291f-4858-a1c1-f66cfb18bc97 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:21.692679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e1687e03-291f-4858-a1c1-f66cfb18bc97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7716a388b6c4b70ac59668e803d7201'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:21.693332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e1687e03-291f-4858-a1c1-f66cfb18bc97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7716a388b6c4b70ac59668e803d7201'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:21.705397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e1687e03-291f-4858-a1c1-f66cfb18bc97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7716a388b6c4b70ac59668e803d7201'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:21.712952 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e1687e03-291f-4858-a1c1-f66cfb18bc97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7716a388b6c4b70ac59668e803d7201'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:21.714186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-e1687e03-291f-4858-a1c1-f66cfb18bc97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7716a388b6c4b70ac59668e803d7201'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:21.726597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1212/2437] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:29:21 2022] DELETE /identity/v3/users/e7716a388b6c4b70ac59668e803d7201 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:29:21.731362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed131a2-0841-4cf9-9b25-63f5aeece230 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:21.732127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed131a2-0841-4cf9-9b25-63f5aeece230 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:21.732305 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed131a2-0841-4cf9-9b25-63f5aeece230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:21.757596 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5ed131a2-0841-4cf9-9b25-63f5aeece230 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:21.758752 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5ed131a2-0841-4cf9-9b25-63f5aeece230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:21.817181 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1226/2438] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:21.827788 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1687e03-291f-4858-a1c1-f66cfb18bc97 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:21.829174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e1687e03-291f-4858-a1c1-f66cfb18bc97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:21.887386 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1687e03-291f-4858-a1c1-f66cfb18bc97 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:21.891164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671934b-a9f1-4ada-bb17-bf792559e306 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:21.891768 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671934b-a9f1-4ada-bb17-bf792559e306 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:21.892056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671934b-a9f1-4ada-bb17-bf792559e306 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:21.892330 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671934b-a9f1-4ada-bb17-bf792559e306 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:21.902175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671934b-a9f1-4ada-bb17-bf792559e306 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:21.903618 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5671934b-a9f1-4ada-bb17-bf792559e306 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:21.913729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1213/2439] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:22.184311 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed131a2-0841-4cf9-9b25-63f5aeece230 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:22.185864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5ed131a2-0841-4cf9-9b25-63f5aeece230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:22.199209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed131a2-0841-4cf9-9b25-63f5aeece230 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:22.204169 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:22.204822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:22.205095 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:22.205366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342 admin admin] PATH_INFO: `/v3/projects/d1610a6082d143d1b0916094775078ee` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:22.206578 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1610a6082d143d1b0916094775078ee)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:22.208544 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:22.257393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1610a6082d143d1b0916094775078ee', 'request_id': 'req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:22.264367 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1610a6082d143d1b0916094775078ee', 'request_id': 'req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:22.271196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1610a6082d143d1b0916094775078ee', 'request_id': 'req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:22.281687 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342 admin admin] The token cache is being invalidate because project d1610a6082d143d1b0916094775078ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:29:22.282020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:22.282929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:22.284458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1227/2440] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:29:22 2022] DELETE /identity/v3/projects/d1610a6082d143d1b0916094775078ee => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:29:26.055335 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72333e25-abe1-4e58-b794-4f84fcf25efe None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:26.055915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72333e25-abe1-4e58-b794-4f84fcf25efe None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:26.055915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72333e25-abe1-4e58-b794-4f84fcf25efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:26.076593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-72333e25-abe1-4e58-b794-4f84fcf25efe None None] MFA Rules not processed for user `22ba5db8353847d7b14ba2d4cb391073`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:26.078086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-72333e25-abe1-4e58-b794-4f84fcf25efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:26.104355 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:26.145372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1214/2441] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:29:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:32.278577 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:36.138555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=15 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:36.894584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:36.896200 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:36.992501 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08c41dc-7b7d-4ff1-a7ef-643c78bb0342 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:36.998314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d7429-64a6-47b9-a2aa-fbbba0cb3af1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:36.998924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d7429-64a6-47b9-a2aa-fbbba0cb3af1 service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:36.999206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d7429-64a6-47b9-a2aa-fbbba0cb3af1 service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:36.999479 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d7429-64a6-47b9-a2aa-fbbba0cb3af1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:37.001248 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f56d7429-64a6-47b9-a2aa-fbbba0cb3af1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:37.012835 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d7429-64a6-47b9-a2aa-fbbba0cb3af1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:37.014442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d7429-64a6-47b9-a2aa-fbbba0cb3af1 service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:37.024822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1228/2442] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:38.720259 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72333e25-abe1-4e58-b794-4f84fcf25efe None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:38.722328 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-72333e25-abe1-4e58-b794-4f84fcf25efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:38.792281 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72333e25-abe1-4e58-b794-4f84fcf25efe None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:38.797769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be0af2-1c4c-4c80-ab89-c284be98c8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:38.798661 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be0af2-1c4c-4c80-ab89-c284be98c8f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:38.798989 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be0af2-1c4c-4c80-ab89-c284be98c8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:38.799317 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be0af2-1c4c-4c80-ab89-c284be98c8f0 admin admin] PATH_INFO: `/v3/users/5f828cb6c46a47628d3190c0f38f92b3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:38.800853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be0af2-1c4c-4c80-ab89-c284be98c8f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f828cb6c46a47628d3190c0f38f92b3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:38.804175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be0af2-1c4c-4c80-ab89-c284be98c8f0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:38.874277 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-90be0af2-1c4c-4c80-ab89-c284be98c8f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f828cb6c46a47628d3190c0f38f92b3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:38.875968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-90be0af2-1c4c-4c80-ab89-c284be98c8f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f828cb6c46a47628d3190c0f38f92b3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:38.894779 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-90be0af2-1c4c-4c80-ab89-c284be98c8f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f828cb6c46a47628d3190c0f38f92b3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:38.905262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-90be0af2-1c4c-4c80-ab89-c284be98c8f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f828cb6c46a47628d3190c0f38f92b3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:38.906243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-90be0af2-1c4c-4c80-ab89-c284be98c8f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f828cb6c46a47628d3190c0f38f92b3'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:38.925298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1215/2443] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:29:38 2022] DELETE /identity/v3/users/5f828cb6c46a47628d3190c0f38f92b3 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:29:38.928744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aef640-b48f-4ba0-91e4-f9e0cbf70fbd None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:38.929224 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aef640-b48f-4ba0-91e4-f9e0cbf70fbd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:38.929538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aef640-b48f-4ba0-91e4-f9e0cbf70fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:38.951119 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a0aef640-b48f-4ba0-91e4-f9e0cbf70fbd None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:38.952540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a0aef640-b48f-4ba0-91e4-f9e0cbf70fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:39.042277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1229/2444] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:29:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:29:39.050415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be0af2-1c4c-4c80-ab89-c284be98c8f0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:39.051859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-90be0af2-1c4c-4c80-ab89-c284be98c8f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:39.132405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be0af2-1c4c-4c80-ab89-c284be98c8f0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:39.136790 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81c452a-511f-4744-ac12-92c02ada3faf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:39.137461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c452a-511f-4744-ac12-92c02ada3faf service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:39.137747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c452a-511f-4744-ac12-92c02ada3faf service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:39.138053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81c452a-511f-4744-ac12-92c02ada3faf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:39.151007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c452a-511f-4744-ac12-92c02ada3faf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:39.152878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81c452a-511f-4744-ac12-92c02ada3faf service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:39.169766 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1216/2445] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:39.463316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aef640-b48f-4ba0-91e4-f9e0cbf70fbd None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:39.464890 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a0aef640-b48f-4ba0-91e4-f9e0cbf70fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:39.480129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aef640-b48f-4ba0-91e4-f9e0cbf70fbd None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:39.485195 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5398cbf7-bcd5-4190-bc8b-d9209d81e067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:39.485839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398cbf7-bcd5-4190-bc8b-d9209d81e067 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:39.486131 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398cbf7-bcd5-4190-bc8b-d9209d81e067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:39.486652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5398cbf7-bcd5-4190-bc8b-d9209d81e067 admin admin] PATH_INFO: `/v3/projects/d1651f8461a34e1ca016176e7dfb8469` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:39.488003 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398cbf7-bcd5-4190-bc8b-d9209d81e067 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1651f8461a34e1ca016176e7dfb8469)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:39.490047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5398cbf7-bcd5-4190-bc8b-d9209d81e067 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:39.541895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5398cbf7-bcd5-4190-bc8b-d9209d81e067 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1651f8461a34e1ca016176e7dfb8469', 'request_id': 'req-5398cbf7-bcd5-4190-bc8b-d9209d81e067'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:39.548237 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5398cbf7-bcd5-4190-bc8b-d9209d81e067 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1651f8461a34e1ca016176e7dfb8469', 'request_id': 'req-5398cbf7-bcd5-4190-bc8b-d9209d81e067'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:39.555598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5398cbf7-bcd5-4190-bc8b-d9209d81e067 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1651f8461a34e1ca016176e7dfb8469', 'request_id': 'req-5398cbf7-bcd5-4190-bc8b-d9209d81e067'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:39.566310 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5398cbf7-bcd5-4190-bc8b-d9209d81e067 admin admin] The token cache is being invalidate because project d1651f8461a34e1ca016176e7dfb8469 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:29:39.566599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5398cbf7-bcd5-4190-bc8b-d9209d81e067 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:39.567224 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-5398cbf7-bcd5-4190-bc8b-d9209d81e067 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:39.568746 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1230/2446] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:29:39 2022] DELETE /identity/v3/projects/d1651f8461a34e1ca016176e7dfb8469 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:29:39.574427 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501338c-dac6-4df6-8e08-f65de59b0b0c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:39.574714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501338c-dac6-4df6-8e08-f65de59b0b0c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:39.574988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d501338c-dac6-4df6-8e08-f65de59b0b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:39.594857 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d501338c-dac6-4df6-8e08-f65de59b0b0c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:39.596097 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d501338c-dac6-4df6-8e08-f65de59b0b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:39.613019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:39.661054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1217/2447] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:39.670046 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03d54f-36c4-4a99-bf65-114233abfecd None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:39.670353 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03d54f-36c4-4a99-bf65-114233abfecd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:39.670626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03d54f-36c4-4a99-bf65-114233abfecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:39.690531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-fd03d54f-36c4-4a99-bf65-114233abfecd None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:39.691733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fd03d54f-36c4-4a99-bf65-114233abfecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:39.700158 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1231/2448] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:29:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:39.705581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501338c-dac6-4df6-8e08-f65de59b0b0c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:39.715456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d501338c-dac6-4df6-8e08-f65de59b0b0c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:39.720100 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede3d89-5d77-4519-9053-02d812192391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:39.720747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede3d89-5d77-4519-9053-02d812192391 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:39.721300 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede3d89-5d77-4519-9053-02d812192391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:39.721300 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede3d89-5d77-4519-9053-02d812192391 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:39.721897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede3d89-5d77-4519-9053-02d812192391 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:29:39.722196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede3d89-5d77-4519-9053-02d812192391 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:39.723812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede3d89-5d77-4519-9053-02d812192391 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:39.735879 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1218/2449] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:29:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:29:39.740296 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03d54f-36c4-4a99-bf65-114233abfecd None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:39.750520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03d54f-36c4-4a99-bf65-114233abfecd None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:39.755357 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de886ffe-7d7d-41cf-9b89-cd6d0babc72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:39.755962 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de886ffe-7d7d-41cf-9b89-cd6d0babc72e admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:39.756235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de886ffe-7d7d-41cf-9b89-cd6d0babc72e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:39.756500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de886ffe-7d7d-41cf-9b89-cd6d0babc72e admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:39.757132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de886ffe-7d7d-41cf-9b89-cd6d0babc72e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:39.759075 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de886ffe-7d7d-41cf-9b89-cd6d0babc72e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:39.763848 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:39.793291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1232/2450] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:29:39 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:29:39.798554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede3d89-5d77-4519-9053-02d812192391 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:39.809078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede3d89-5d77-4519-9053-02d812192391 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:39.865028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc8202-7591-45bf-8526-01f8957005b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:39.865637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc8202-7591-45bf-8526-01f8957005b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:39.865637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc8202-7591-45bf-8526-01f8957005b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:39.865637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffc8202-7591-45bf-8526-01f8957005b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:39.866441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc8202-7591-45bf-8526-01f8957005b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:39.868156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffc8202-7591-45bf-8526-01f8957005b7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:39.890466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1219/2451] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:29:39 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:29:39.895145 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de886ffe-7d7d-41cf-9b89-cd6d0babc72e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:39.906188 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de886ffe-7d7d-41cf-9b89-cd6d0babc72e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:39.911316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c10276-8db8-4642-aadf-9daa358251bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:39.911916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c10276-8db8-4642-aadf-9daa358251bb admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:39.912194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c10276-8db8-4642-aadf-9daa358251bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:39.912465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c10276-8db8-4642-aadf-9daa358251bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:39.913108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c10276-8db8-4642-aadf-9daa358251bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:39.914730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c10276-8db8-4642-aadf-9daa358251bb admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:39.925639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1233/2452] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:29:39.929793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc8202-7591-45bf-8526-01f8957005b7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:39.940463 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffc8202-7591-45bf-8526-01f8957005b7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:39.946108 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7982889-c6d5-491f-b85f-e52954546a21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:39.946819 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982889-c6d5-491f-b85f-e52954546a21 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:39.947133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982889-c6d5-491f-b85f-e52954546a21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:39.947402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982889-c6d5-491f-b85f-e52954546a21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:39.948174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7982889-c6d5-491f-b85f-e52954546a21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:39.949747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7982889-c6d5-491f-b85f-e52954546a21 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:39.961146 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1220/2453] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:29:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:29:39.966306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c10276-8db8-4642-aadf-9daa358251bb admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:39.976462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c10276-8db8-4642-aadf-9daa358251bb admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:39.981579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebda64f-d4c7-4737-b5a5-9306d822e000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:39.982237 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebda64f-d4c7-4737-b5a5-9306d822e000 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:39.982520 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebda64f-d4c7-4737-b5a5-9306d822e000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:39.982789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebda64f-d4c7-4737-b5a5-9306d822e000 admin admin] PATH_INFO: `/v3/projects/60386a4f71754e4396434dfa34c55b75/users/24396d51e9d243f7920f9fc309d66115/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:39.999661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebda64f-d4c7-4737-b5a5-9306d822e000 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60386a4f71754e4396434dfa34c55b75, user_id=24396d51e9d243f7920f9fc309d66115, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:40.002345 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebda64f-d4c7-4737-b5a5-9306d822e000 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:40.016133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1234/2454] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:29:39 2022] PUT /identity/v3/projects/60386a4f71754e4396434dfa34c55b75/users/24396d51e9d243f7920f9fc309d66115/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:29:40.021254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea10af-de66-46ce-a75c-01e7cf253749 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:40.021542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea10af-de66-46ce-a75c-01e7cf253749 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:40.021806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ea10af-de66-46ce-a75c-01e7cf253749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:40.042876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c1ea10af-de66-46ce-a75c-01e7cf253749 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:40.044136 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c1ea10af-de66-46ce-a75c-01e7cf253749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:40.105720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1221/2455] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:29:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:40.115193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebda64f-d4c7-4737-b5a5-9306d822e000 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:40.116956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7ebda64f-d4c7-4737-b5a5-9306d822e000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:40.173810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebda64f-d4c7-4737-b5a5-9306d822e000 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:40.177470 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e371c-571b-4914-b4a4-043bbc39d335 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:40.178073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e371c-571b-4914-b4a4-043bbc39d335 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:40.178380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e371c-571b-4914-b4a4-043bbc39d335 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:40.178654 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250e371c-571b-4914-b4a4-043bbc39d335 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:40.188244 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e371c-571b-4914-b4a4-043bbc39d335 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:40.189772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250e371c-571b-4914-b4a4-043bbc39d335 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:40.200724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1235/2456] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:45.883976 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d00c1f-4b38-4640-adad-1e5fef72f09d None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:45.884999 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d00c1f-4b38-4640-adad-1e5fef72f09d None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:45.885125 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d00c1f-4b38-4640-adad-1e5fef72f09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:45.929567 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d1d00c1f-4b38-4640-adad-1e5fef72f09d None None] MFA Rules not processed for user `24396d51e9d243f7920f9fc309d66115`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:45.930884 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d1d00c1f-4b38-4640-adad-1e5fef72f09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:45.963777 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:46.025812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1222/2457] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:29:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:46.030093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e371c-571b-4914-b4a4-043bbc39d335 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:46.040755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250e371c-571b-4914-b4a4-043bbc39d335 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:46.045551 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c77d0-666b-4fb2-abb0-01c6bc21e1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:46.046301 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c77d0-666b-4fb2-abb0-01c6bc21e1f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:46.046785 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c77d0-666b-4fb2-abb0-01c6bc21e1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:46.047249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347c77d0-666b-4fb2-abb0-01c6bc21e1f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:46.048049 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c77d0-666b-4fb2-abb0-01c6bc21e1f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:46.050126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347c77d0-666b-4fb2-abb0-01c6bc21e1f7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:46.083175 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1236/2458] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:29:46 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:29:46.088133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d00c1f-4b38-4640-adad-1e5fef72f09d None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:46.105013 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d00c1f-4b38-4640-adad-1e5fef72f09d None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:46.176245 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd1b44-9d8e-487c-8812-b0efdf38023b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:46.176877 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd1b44-9d8e-487c-8812-b0efdf38023b admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:46.177151 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd1b44-9d8e-487c-8812-b0efdf38023b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:46.177420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd1b44-9d8e-487c-8812-b0efdf38023b admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:46.178053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd1b44-9d8e-487c-8812-b0efdf38023b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:46.180144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd1b44-9d8e-487c-8812-b0efdf38023b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:46.210393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1223/2459] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:29:46 2022] POST /identity/v3/users => generated 298 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:29:46.212766 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c77d0-666b-4fb2-abb0-01c6bc21e1f7 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:46.225232 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347c77d0-666b-4fb2-abb0-01c6bc21e1f7 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:46.230814 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f198749a-80ff-4c5e-97bb-807078a127d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:46.231546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198749a-80ff-4c5e-97bb-807078a127d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:46.231923 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198749a-80ff-4c5e-97bb-807078a127d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:46.232280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f198749a-80ff-4c5e-97bb-807078a127d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:46.233124 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f198749a-80ff-4c5e-97bb-807078a127d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:46.235247 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f198749a-80ff-4c5e-97bb-807078a127d7 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:46.248023 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1237/2460] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:29:46.252508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd1b44-9d8e-487c-8812-b0efdf38023b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:46.263817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd1b44-9d8e-487c-8812-b0efdf38023b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:46.269057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95e6f4-9b75-4d6a-b7e6-68eebf1c6904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:46.270478 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95e6f4-9b75-4d6a-b7e6-68eebf1c6904 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:46.270763 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95e6f4-9b75-4d6a-b7e6-68eebf1c6904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:46.271035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95e6f4-9b75-4d6a-b7e6-68eebf1c6904 admin admin] PATH_INFO: `/v3/projects/27b2d0c6cccf4117ab4e7e7ea3ce8f07/users/ca78e1925cd54b679ddd1406aaf4d9b2/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:46.287732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95e6f4-9b75-4d6a-b7e6-68eebf1c6904 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b2d0c6cccf4117ab4e7e7ea3ce8f07, user_id=ca78e1925cd54b679ddd1406aaf4d9b2, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:46.290484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95e6f4-9b75-4d6a-b7e6-68eebf1c6904 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:46.305810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1224/2461] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:29:46 2022] PUT /identity/v3/projects/27b2d0c6cccf4117ab4e7e7ea3ce8f07/users/ca78e1925cd54b679ddd1406aaf4d9b2/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:29:46.310319 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f198749a-80ff-4c5e-97bb-807078a127d7 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:46.325425 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f198749a-80ff-4c5e-97bb-807078a127d7 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:46.392281 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63137e2c-a93e-4d83-ab74-9fe2e196cf9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:46.393020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63137e2c-a93e-4d83-ab74-9fe2e196cf9f admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:46.393299 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63137e2c-a93e-4d83-ab74-9fe2e196cf9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:46.393572 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63137e2c-a93e-4d83-ab74-9fe2e196cf9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:46.394235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63137e2c-a93e-4d83-ab74-9fe2e196cf9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:46.395965 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63137e2c-a93e-4d83-ab74-9fe2e196cf9f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:46.407136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1238/2462] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:29:46 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:29:46.411751 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95e6f4-9b75-4d6a-b7e6-68eebf1c6904 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:46.422766 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95e6f4-9b75-4d6a-b7e6-68eebf1c6904 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:46.427441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bff60-a18a-40ea-a21a-a67956c6a207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:46.428043 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bff60-a18a-40ea-a21a-a67956c6a207 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:46.428317 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bff60-a18a-40ea-a21a-a67956c6a207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:46.428584 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bff60-a18a-40ea-a21a-a67956c6a207 admin admin] PATH_INFO: `/v3/projects/27b2d0c6cccf4117ab4e7e7ea3ce8f07/users/ca78e1925cd54b679ddd1406aaf4d9b2/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:46.430506 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397bff60-a18a-40ea-a21a-a67956c6a207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27b2d0c6cccf4117ab4e7e7ea3ce8f07, user_id=ca78e1925cd54b679ddd1406aaf4d9b2, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:46.433714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397bff60-a18a-40ea-a21a-a67956c6a207 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:46.448154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1225/2463] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:29:46 2022] PUT /identity/v3/projects/27b2d0c6cccf4117ab4e7e7ea3ce8f07/users/ca78e1925cd54b679ddd1406aaf4d9b2/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:29:51.183846 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63137e2c-a93e-4d83-ab74-9fe2e196cf9f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:51.185684 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-63137e2c-a93e-4d83-ab74-9fe2e196cf9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:51.206192 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63137e2c-a93e-4d83-ab74-9fe2e196cf9f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:51.231449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:51.330746 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765093a3-a40c-4961-bbd4-b575f29b2dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:51.331818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765093a3-a40c-4961-bbd4-b575f29b2dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:51.332248 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765093a3-a40c-4961-bbd4-b575f29b2dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:51.332782 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765093a3-a40c-4961-bbd4-b575f29b2dc4 admin admin] PATH_INFO: `/v3/users/f5eebe729eee4bf4b41f4de32e5a44c9` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:51.334038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765093a3-a40c-4961-bbd4-b575f29b2dc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5eebe729eee4bf4b41f4de32e5a44c9)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:51.336165 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765093a3-a40c-4961-bbd4-b575f29b2dc4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:51.416178 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-765093a3-a40c-4961-bbd4-b575f29b2dc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5eebe729eee4bf4b41f4de32e5a44c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:51.430263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-765093a3-a40c-4961-bbd4-b575f29b2dc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5eebe729eee4bf4b41f4de32e5a44c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:51.446352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-765093a3-a40c-4961-bbd4-b575f29b2dc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5eebe729eee4bf4b41f4de32e5a44c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:51.447699 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-765093a3-a40c-4961-bbd4-b575f29b2dc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5eebe729eee4bf4b41f4de32e5a44c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:51.462894 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-765093a3-a40c-4961-bbd4-b575f29b2dc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5eebe729eee4bf4b41f4de32e5a44c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:51.473315 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1239/2464] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:29:51 2022] DELETE /identity/v3/users/f5eebe729eee4bf4b41f4de32e5a44c9 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:29:51.477839 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33d705-e97b-454d-997a-de44a5f3a5ff None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:51.478363 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33d705-e97b-454d-997a-de44a5f3a5ff None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:51.478474 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c33d705-e97b-454d-997a-de44a5f3a5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:51.511847 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5c33d705-e97b-454d-997a-de44a5f3a5ff None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:51.513109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5c33d705-e97b-454d-997a-de44a5f3a5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:51.601759 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1226/2465] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:51.614691 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765093a3-a40c-4961-bbd4-b575f29b2dc4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:51.616137 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-765093a3-a40c-4961-bbd4-b575f29b2dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:51.736806 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765093a3-a40c-4961-bbd4-b575f29b2dc4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:51.740926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570b54e-5b15-4388-aee6-42c524011d74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:51.741644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570b54e-5b15-4388-aee6-42c524011d74 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:51.742018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570b54e-5b15-4388-aee6-42c524011d74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:51.742398 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570b54e-5b15-4388-aee6-42c524011d74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:51.755023 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570b54e-5b15-4388-aee6-42c524011d74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:51.757160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570b54e-5b15-4388-aee6-42c524011d74 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:51.776599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1240/2466] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:51.797780 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa83b02-5115-4c48-8d9a-15fcf61d62bc None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:51.797780 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa83b02-5115-4c48-8d9a-15fcf61d62bc None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:51.797780 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa83b02-5115-4c48-8d9a-15fcf61d62bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:51.825123 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:51.838142 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7fa83b02-5115-4c48-8d9a-15fcf61d62bc None None] MFA Rules not processed for user `ca78e1925cd54b679ddd1406aaf4d9b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:51.839867 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7fa83b02-5115-4c48-8d9a-15fcf61d62bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:51.972583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1227/2467] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:29:51 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:29:51.981560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570b54e-5b15-4388-aee6-42c524011d74 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:51.983118 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6570b54e-5b15-4388-aee6-42c524011d74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:52.039075 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:29:52.059773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570b54e-5b15-4388-aee6-42c524011d74 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:52.065317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ab654-b1e1-4b49-8e5b-e143b67f0231 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96c71ea88d9a474c9ff7a30555cfa91f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:52.066048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ab654-b1e1-4b49-8e5b-e143b67f0231 service glance] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:52.066561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ab654-b1e1-4b49-8e5b-e143b67f0231 service glance] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:52.066834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ab654-b1e1-4b49-8e5b-e143b67f0231 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:52.068792 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-275ab654-b1e1-4b49-8e5b-e143b67f0231 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:52.138025 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa83b02-5115-4c48-8d9a-15fcf61d62bc None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:52.140676 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7fa83b02-5115-4c48-8d9a-15fcf61d62bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:52.144461 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ab654-b1e1-4b49-8e5b-e143b67f0231 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:52.145968 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ab654-b1e1-4b49-8e5b-e143b67f0231 service glance] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:52.156858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1241/2468] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:29:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:29:52.180058 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa83b02-5115-4c48-8d9a-15fcf61d62bc None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:52.189311 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29abab-6745-4d5a-b850-46481eb5ddff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:52.190345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29abab-6745-4d5a-b850-46481eb5ddff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:52.190803 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29abab-6745-4d5a-b850-46481eb5ddff admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:52.191244 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29abab-6745-4d5a-b850-46481eb5ddff admin admin] PATH_INFO: `/v3/projects/8fcdf2893664457ba53cc8a4793f4d53` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:52.193393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f29abab-6745-4d5a-b850-46481eb5ddff admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fcdf2893664457ba53cc8a4793f4d53)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:52.199030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f29abab-6745-4d5a-b850-46481eb5ddff admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:52.311720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4f29abab-6745-4d5a-b850-46481eb5ddff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fcdf2893664457ba53cc8a4793f4d53', 'request_id': 'req-4f29abab-6745-4d5a-b850-46481eb5ddff'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:52.318137 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4f29abab-6745-4d5a-b850-46481eb5ddff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fcdf2893664457ba53cc8a4793f4d53', 'request_id': 'req-4f29abab-6745-4d5a-b850-46481eb5ddff'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:52.338669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4f29abab-6745-4d5a-b850-46481eb5ddff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fcdf2893664457ba53cc8a4793f4d53', 'request_id': 'req-4f29abab-6745-4d5a-b850-46481eb5ddff'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:52.366413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10910a5-202d-4c71-a8cd-d1ad9c14990b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:52.366413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10910a5-202d-4c71-a8cd-d1ad9c14990b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:52.366413 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10910a5-202d-4c71-a8cd-d1ad9c14990b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:52.372030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4f29abab-6745-4d5a-b850-46481eb5ddff admin admin] The token cache is being invalidate because project 8fcdf2893664457ba53cc8a4793f4d53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:29:52.372326 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4f29abab-6745-4d5a-b850-46481eb5ddff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:52.375309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-4f29abab-6745-4d5a-b850-46481eb5ddff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:29:52.379688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1228/2469] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:29:52 2022] DELETE /identity/v3/projects/8fcdf2893664457ba53cc8a4793f4d53 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:29:52.399024 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b10910a5-202d-4c71-a8cd-d1ad9c14990b None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:29:52.400777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b10910a5-202d-4c71-a8cd-d1ad9c14990b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:52.519658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1242/2470] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 15:29:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:29:52.585669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29abab-6745-4d5a-b850-46481eb5ddff admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:29:52.587263 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4f29abab-6745-4d5a-b850-46481eb5ddff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:29:52.674719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29abab-6745-4d5a-b850-46481eb5ddff admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:29:52.681864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a6e00-d092-42bb-9ed1-186edec2784a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:29:52.682722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a6e00-d092-42bb-9ed1-186edec2784a service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:29:52.682999 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a6e00-d092-42bb-9ed1-186edec2784a service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:29:52.683278 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a6e00-d092-42bb-9ed1-186edec2784a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:29:52.704917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a6e00-d092-42bb-9ed1-186edec2784a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:29:52.707902 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a6e00-d092-42bb-9ed1-186edec2784a service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:29:52.719942 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1229/2471] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:29:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:30:02.505971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=56 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:02.654878 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0040e-1d40-4ae6-9ef8-4a5034b5b970 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:02.669040 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0040e-1d40-4ae6-9ef8-4a5034b5b970 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:02.669433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0040e-1d40-4ae6-9ef8-4a5034b5b970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:02.689972 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b5d0040e-1d40-4ae6-9ef8-4a5034b5b970 None None] MFA Rules not processed for user `24396d51e9d243f7920f9fc309d66115`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:02.691539 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b5d0040e-1d40-4ae6-9ef8-4a5034b5b970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:02.714177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:02.751587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1243/2472] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:30:02 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:11.185862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa8fd5-00d1-4048-8645-02918dfdc263 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:11.186508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa8fd5-00d1-4048-8645-02918dfdc263 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:11.186508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fa8fd5-00d1-4048-8645-02918dfdc263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:11.213088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-f7fa8fd5-00d1-4048-8645-02918dfdc263 None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:11.214598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f7fa8fd5-00d1-4048-8645-02918dfdc263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:11.238412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1230/2473] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 15:30:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:11.240461 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0040e-1d40-4ae6-9ef8-4a5034b5b970 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:11.254466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d0040e-1d40-4ae6-9ef8-4a5034b5b970 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:11.259821 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08b1e18-e812-4ebe-8c3a-018f277566bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:11.260459 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b1e18-e812-4ebe-8c3a-018f277566bd service swift] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:11.260738 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b1e18-e812-4ebe-8c3a-018f277566bd service swift] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:11.261226 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08b1e18-e812-4ebe-8c3a-018f277566bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:11.270711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b1e18-e812-4ebe-8c3a-018f277566bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:11.272422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08b1e18-e812-4ebe-8c3a-018f277566bd service swift] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:11.283691 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1244/2474] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:30:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:30:11.718395 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb1e8b-4cc3-497d-bd61-7e83cf87ea29 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:11.719000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb1e8b-4cc3-497d-bd61-7e83cf87ea29 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:11.719498 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fb1e8b-4cc3-497d-bd61-7e83cf87ea29 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:11.721763 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1231/2475] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 15:30:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:30:11.728237 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53337bd-99a4-4d18-884d-123f248935e3 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:11.729157 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53337bd-99a4-4d18-884d-123f248935e3 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:11.729675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53337bd-99a4-4d18-884d-123f248935e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:11.756450 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f53337bd-99a4-4d18-884d-123f248935e3 None None] MFA Rules not processed for user `8a5fd31a2db944f596912e4f0aa26835`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:11.758091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f53337bd-99a4-4d18-884d-123f248935e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:11.906123 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1245/2476] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 15:30:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:30:12.051450 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb1e8b-4cc3-497d-bd61-7e83cf87ea29 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:12.053345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-12fb1e8b-4cc3-497d-bd61-7e83cf87ea29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:12.081507 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fb1e8b-4cc3-497d-bd61-7e83cf87ea29 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:12.092553 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27752e72-fc9e-4bfb-b3bf-673db4ea20c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:12.094202 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27752e72-fc9e-4bfb-b3bf-673db4ea20c5 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:12.094729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27752e72-fc9e-4bfb-b3bf-673db4ea20c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:12.095238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27752e72-fc9e-4bfb-b3bf-673db4ea20c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:12.113650 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27752e72-fc9e-4bfb-b3bf-673db4ea20c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:12.115293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27752e72-fc9e-4bfb-b3bf-673db4ea20c5 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:12.130554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1232/2477] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:30:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:30:21.875360 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=62 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:22.122178 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=9 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:38.684076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db9756-1dc0-4562-ab0a-2b543df2a9d8 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:38.684559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db9756-1dc0-4562-ab0a-2b543df2a9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:38.684683 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08db9756-1dc0-4562-ab0a-2b543df2a9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:38.703944 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-08db9756-1dc0-4562-ab0a-2b543df2a9d8 None None] MFA Rules not processed for user `22c5f040350d4daba0ebe2af86849a23`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:38.705138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-08db9756-1dc0-4562-ab0a-2b543df2a9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:38.759034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1246/2478] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:30:38 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:44.042178 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54351bb0-2901-4d33-98db-0af12c3b5af0 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:44.042658 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54351bb0-2901-4d33-98db-0af12c3b5af0 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:44.042749 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54351bb0-2901-4d33-98db-0af12c3b5af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:44.061875 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-54351bb0-2901-4d33-98db-0af12c3b5af0 None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:44.063103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-54351bb0-2901-4d33-98db-0af12c3b5af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:44.070740 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1233/2479] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 15:30:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:30:44.084508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db9756-1dc0-4562-ab0a-2b543df2a9d8 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:30:44.096217 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08db9756-1dc0-4562-ab0a-2b543df2a9d8 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:30:44.101379 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b3b81a-7f3a-4da0-be2d-b906bec9f4bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:30:44.101974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3b81a-7f3a-4da0-be2d-b906bec9f4bd service swift] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:44.102279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3b81a-7f3a-4da0-be2d-b906bec9f4bd service swift] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:44.102557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b3b81a-7f3a-4da0-be2d-b906bec9f4bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:44.110702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3b81a-7f3a-4da0-be2d-b906bec9f4bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:30:44.112282 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b3b81a-7f3a-4da0-be2d-b906bec9f4bd service swift] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:30:44.120707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1247/2480] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:30:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:30:54.057405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:54.116974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=31 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:30:57.066367 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eadaac-1c68-466c-9c99-3a4939a2cb42 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:30:57.067293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eadaac-1c68-466c-9c99-3a4939a2cb42 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:30:57.067739 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eadaac-1c68-466c-9c99-3a4939a2cb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:30:57.092188 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a5eadaac-1c68-466c-9c99-3a4939a2cb42 None None] MFA Rules not processed for user `461b9a3987744067a62234aaa8d086d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:30:57.093885 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a5eadaac-1c68-466c-9c99-3a4939a2cb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:30:57.166437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1234/2481] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:30:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:01.671571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ff9a0-9816-40aa-9d52-28489f4048b4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:01.672141 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ff9a0-9816-40aa-9d52-28489f4048b4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:01.672141 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899ff9a0-9816-40aa-9d52-28489f4048b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:01.691482 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-899ff9a0-9816-40aa-9d52-28489f4048b4 None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:01.692691 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-899ff9a0-9816-40aa-9d52-28489f4048b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:01.700065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1248/2482] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 24 15:31:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:31:01.712642 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eadaac-1c68-466c-9c99-3a4939a2cb42 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:01.724417 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eadaac-1c68-466c-9c99-3a4939a2cb42 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:01.729808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96df02-b9cb-4fb2-b09e-7531031a3679 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:01.730426 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96df02-b9cb-4fb2-b09e-7531031a3679 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:01.730698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96df02-b9cb-4fb2-b09e-7531031a3679 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:01.730990 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f96df02-b9cb-4fb2-b09e-7531031a3679 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:01.739263 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96df02-b9cb-4fb2-b09e-7531031a3679 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:01.740836 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f96df02-b9cb-4fb2-b09e-7531031a3679 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:01.750056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1235/2483] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:31:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:31:03.745261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ff9a0-9816-40aa-9d52-28489f4048b4 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:03.747283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-899ff9a0-9816-40aa-9d52-28489f4048b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:03.814602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899ff9a0-9816-40aa-9d52-28489f4048b4 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:03.819086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7737bc2-c4b1-4e09-80a6-4641ab1c53a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:03.819708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7737bc2-c4b1-4e09-80a6-4641ab1c53a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:03.819981 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7737bc2-c4b1-4e09-80a6-4641ab1c53a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:03.820248 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7737bc2-c4b1-4e09-80a6-4641ab1c53a1 admin admin] PATH_INFO: `/v3/users/22ba5db8353847d7b14ba2d4cb391073` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:03.821362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7737bc2-c4b1-4e09-80a6-4641ab1c53a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22ba5db8353847d7b14ba2d4cb391073)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:03.823327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7737bc2-c4b1-4e09-80a6-4641ab1c53a1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:03.868602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f7737bc2-c4b1-4e09-80a6-4641ab1c53a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22ba5db8353847d7b14ba2d4cb391073'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:31:03.876058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f7737bc2-c4b1-4e09-80a6-4641ab1c53a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22ba5db8353847d7b14ba2d4cb391073'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:31:03.888194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f7737bc2-c4b1-4e09-80a6-4641ab1c53a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22ba5db8353847d7b14ba2d4cb391073'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:31:03.888878 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f7737bc2-c4b1-4e09-80a6-4641ab1c53a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22ba5db8353847d7b14ba2d4cb391073'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:31:03.897154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-f7737bc2-c4b1-4e09-80a6-4641ab1c53a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22ba5db8353847d7b14ba2d4cb391073'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:31:03.905703 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1249/2484] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:31:03 2022] DELETE /identity/v3/users/22ba5db8353847d7b14ba2d4cb391073 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:31:04.178101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96df02-b9cb-4fb2-b09e-7531031a3679 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:04.211096 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f96df02-b9cb-4fb2-b09e-7531031a3679 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:04.286697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f94d47-bb4f-4e39-ae8a-9c13ecc84565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:04.287458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f94d47-bb4f-4e39-ae8a-9c13ecc84565 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:04.287860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f94d47-bb4f-4e39-ae8a-9c13ecc84565 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:04.288637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f94d47-bb4f-4e39-ae8a-9c13ecc84565 admin admin] PATH_INFO: `/v3/projects/7292443f39f548e79c3bb9d07bccb0df` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:04.289856 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f94d47-bb4f-4e39-ae8a-9c13ecc84565 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7292443f39f548e79c3bb9d07bccb0df)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:04.291782 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f94d47-bb4f-4e39-ae8a-9c13ecc84565 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:04.340522 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41f94d47-bb4f-4e39-ae8a-9c13ecc84565 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7292443f39f548e79c3bb9d07bccb0df', 'request_id': 'req-41f94d47-bb4f-4e39-ae8a-9c13ecc84565'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:31:04.347670 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41f94d47-bb4f-4e39-ae8a-9c13ecc84565 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7292443f39f548e79c3bb9d07bccb0df', 'request_id': 'req-41f94d47-bb4f-4e39-ae8a-9c13ecc84565'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:31:04.374550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41f94d47-bb4f-4e39-ae8a-9c13ecc84565 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7292443f39f548e79c3bb9d07bccb0df', 'request_id': 'req-41f94d47-bb4f-4e39-ae8a-9c13ecc84565'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:31:04.388860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41f94d47-bb4f-4e39-ae8a-9c13ecc84565 admin admin] The token cache is being invalidate because project 7292443f39f548e79c3bb9d07bccb0df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:31:04.389155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41f94d47-bb4f-4e39-ae8a-9c13ecc84565 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:31:04.389803 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41f94d47-bb4f-4e39-ae8a-9c13ecc84565 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:31:04.391917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1236/2485] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:31:04 2022] DELETE /identity/v3/projects/7292443f39f548e79c3bb9d07bccb0df => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:31:04.396704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997a6db-0c40-41cb-9c21-a2963c44bbe6 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:04.396990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997a6db-0c40-41cb-9c21-a2963c44bbe6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:04.397798 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997a6db-0c40-41cb-9c21-a2963c44bbe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:04.419941 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0997a6db-0c40-41cb-9c21-a2963c44bbe6 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:04.421194 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0997a6db-0c40-41cb-9c21-a2963c44bbe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:04.525332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1250/2486] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:31:04.535325 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788615f9-d3ec-4965-9d3a-ceaab4af490c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:04.535610 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788615f9-d3ec-4965-9d3a-ceaab4af490c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:04.535926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788615f9-d3ec-4965-9d3a-ceaab4af490c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:04.559573 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-788615f9-d3ec-4965-9d3a-ceaab4af490c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:04.560852 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-788615f9-d3ec-4965-9d3a-ceaab4af490c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:04.573437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1237/2487] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:04.582272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997a6db-0c40-41cb-9c21-a2963c44bbe6 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:04.587929 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997a6db-0c40-41cb-9c21-a2963c44bbe6 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:04.592750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f764e8-f9ec-43cd-a055-281d8187fdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:04.593358 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f764e8-f9ec-43cd-a055-281d8187fdd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:04.593639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f764e8-f9ec-43cd-a055-281d8187fdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:04.593914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f764e8-f9ec-43cd-a055-281d8187fdd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:04.594618 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f764e8-f9ec-43cd-a055-281d8187fdd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:31:04.594922 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f764e8-f9ec-43cd-a055-281d8187fdd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:04.596659 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f764e8-f9ec-43cd-a055-281d8187fdd3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:04.610142 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:04.614721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1251/2488] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:31:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:31:04.620493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788615f9-d3ec-4965-9d3a-ceaab4af490c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:04.633407 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788615f9-d3ec-4965-9d3a-ceaab4af490c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:04.641718 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbded2-7c21-4cd7-9277-849aa8e73f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:04.642373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbded2-7c21-4cd7-9277-849aa8e73f4d admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:04.642657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbded2-7c21-4cd7-9277-849aa8e73f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:04.642925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fbded2-7c21-4cd7-9277-849aa8e73f4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:04.643554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbded2-7c21-4cd7-9277-849aa8e73f4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:04.645833 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fbded2-7c21-4cd7-9277-849aa8e73f4d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:04.676267 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:04.701254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1238/2489] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:31:04 2022] POST /identity/v3/projects => generated 369 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:31:04.706068 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f764e8-f9ec-43cd-a055-281d8187fdd3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:04.719465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f764e8-f9ec-43cd-a055-281d8187fdd3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:04.802475 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e95339-ca25-4225-8f13-c0441f4e6882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:04.804606 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e95339-ca25-4225-8f13-c0441f4e6882 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:04.805087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e95339-ca25-4225-8f13-c0441f4e6882 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:04.806538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e95339-ca25-4225-8f13-c0441f4e6882 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:04.807525 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e95339-ca25-4225-8f13-c0441f4e6882 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:04.809704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e95339-ca25-4225-8f13-c0441f4e6882 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:04.845331 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbded2-7c21-4cd7-9277-849aa8e73f4d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:04.845962 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1252/2490] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:31:04 2022] POST /identity/v3/users => generated 285 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:31:04.847401 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e95339-ca25-4225-8f13-c0441f4e6882 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:04.847769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-69fbded2-7c21-4cd7-9277-849aa8e73f4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:04.865303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e95339-ca25-4225-8f13-c0441f4e6882 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:04.881306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76ad9e-29e6-423a-a94d-e17275089745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:04.882252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76ad9e-29e6-423a-a94d-e17275089745 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:04.882556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76ad9e-29e6-423a-a94d-e17275089745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:04.882838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76ad9e-29e6-423a-a94d-e17275089745 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:04.883477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76ad9e-29e6-423a-a94d-e17275089745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:04.885792 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76ad9e-29e6-423a-a94d-e17275089745 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:04.899271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1253/2491] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:31:04.906953 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76ad9e-29e6-423a-a94d-e17275089745 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:04.919541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76ad9e-29e6-423a-a94d-e17275089745 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:04.925248 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaaa998-33d7-4284-afca-8c23abe40c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:04.926125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaa998-33d7-4284-afca-8c23abe40c1c admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:04.926455 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaa998-33d7-4284-afca-8c23abe40c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:04.926736 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaaa998-33d7-4284-afca-8c23abe40c1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:04.927377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaaa998-33d7-4284-afca-8c23abe40c1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:04.929004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaaa998-33d7-4284-afca-8c23abe40c1c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:04.929839 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fbded2-7c21-4cd7-9277-849aa8e73f4d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:04.935115 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007eea7e-9dac-4603-be3a-136940aa7b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:04.936535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007eea7e-9dac-4603-be3a-136940aa7b8f service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:04.937130 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007eea7e-9dac-4603-be3a-136940aa7b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:04.937908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007eea7e-9dac-4603-be3a-136940aa7b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:04.940054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-007eea7e-9dac-4603-be3a-136940aa7b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:04.941362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1254/2492] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:31:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:31:04.945078 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaaa998-33d7-4284-afca-8c23abe40c1c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:04.957644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaaa998-33d7-4284-afca-8c23abe40c1c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:04.963393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee14cb1-2da3-4b55-8e88-23ad71b401dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:04.964063 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee14cb1-2da3-4b55-8e88-23ad71b401dd admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:04.964342 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee14cb1-2da3-4b55-8e88-23ad71b401dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:04.964620 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee14cb1-2da3-4b55-8e88-23ad71b401dd admin admin] PATH_INFO: `/v3/projects/a17f14aab8fb4fc4b2240906b314a865/users/88709c5fa47f4645a6aeda58cdec4ede/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:04.989904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee14cb1-2da3-4b55-8e88-23ad71b401dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a17f14aab8fb4fc4b2240906b314a865, user_id=88709c5fa47f4645a6aeda58cdec4ede, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:04.993899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee14cb1-2da3-4b55-8e88-23ad71b401dd admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:05.011699 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1255/2493] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:31:04 2022] PUT /identity/v3/projects/a17f14aab8fb4fc4b2240906b314a865/users/88709c5fa47f4645a6aeda58cdec4ede/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:31:05.023752 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda28f5-3944-4d6f-9c8e-083ca9e111e8 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:05.024400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda28f5-3944-4d6f-9c8e-083ca9e111e8 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:05.024673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dda28f5-3944-4d6f-9c8e-083ca9e111e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:05.026892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007eea7e-9dac-4603-be3a-136940aa7b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:05.029337 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007eea7e-9dac-4603-be3a-136940aa7b8f service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:05.046749 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1239/2494] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:04 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:05.059296 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-9dda28f5-3944-4d6f-9c8e-083ca9e111e8 None None] MFA Rules not processed for user `88709c5fa47f4645a6aeda58cdec4ede`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:05.060775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9dda28f5-3944-4d6f-9c8e-083ca9e111e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:05.172779 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1256/2495] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:31:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:05.193443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007eea7e-9dac-4603-be3a-136940aa7b8f service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:05.195519 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-007eea7e-9dac-4603-be3a-136940aa7b8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:05.287964 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007eea7e-9dac-4603-be3a-136940aa7b8f service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:05.401836 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223251a-a023-4f0f-80e6-b4ab7f7596a5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96c71ea88d9a474c9ff7a30555cfa91f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:05.402497 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223251a-a023-4f0f-80e6-b4ab7f7596a5 service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:05.402926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223251a-a023-4f0f-80e6-b4ab7f7596a5 service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:05.403441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d223251a-a023-4f0f-80e6-b4ab7f7596a5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:05.436512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223251a-a023-4f0f-80e6-b4ab7f7596a5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:05.438502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d223251a-a023-4f0f-80e6-b4ab7f7596a5 service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:05.453702 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1240/2496] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:15.167158 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=81,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:15.446271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:INSERT=1,keystone:SELECT=98 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:24.686677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcce75-2b08-44e8-9516-4b226563dd85 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:24.687221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcce75-2b08-44e8-9516-4b226563dd85 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:24.687221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebcce75-2b08-44e8-9516-4b226563dd85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:24.710729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8ebcce75-2b08-44e8-9516-4b226563dd85 None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:24.711973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8ebcce75-2b08-44e8-9516-4b226563dd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:24.777336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1257/2497] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 15:31:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:24.790900 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223251a-a023-4f0f-80e6-b4ab7f7596a5 service glance] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:24.792410 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d223251a-a023-4f0f-80e6-b4ab7f7596a5 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:24.869317 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d223251a-a023-4f0f-80e6-b4ab7f7596a5 service glance] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:24.875965 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb877ff-9513-4ba0-a605-5763a9099e09 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:24.876578 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb877ff-9513-4ba0-a605-5763a9099e09 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:24.876857 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb877ff-9513-4ba0-a605-5763a9099e09 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:24.877127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb877ff-9513-4ba0-a605-5763a9099e09 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:24.886370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb877ff-9513-4ba0-a605-5763a9099e09 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:24.887983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb877ff-9513-4ba0-a605-5763a9099e09 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:24.896593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1241/2498] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:31:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:31:26.391227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcce75-2b08-44e8-9516-4b226563dd85 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:26.393419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8ebcce75-2b08-44e8-9516-4b226563dd85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:26.484481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebcce75-2b08-44e8-9516-4b226563dd85 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:26.488546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f3c63-4a45-461d-9ec4-63f024fb5bff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:26.489249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f3c63-4a45-461d-9ec4-63f024fb5bff service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:26.489592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f3c63-4a45-461d-9ec4-63f024fb5bff service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:26.490027 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f3c63-4a45-461d-9ec4-63f024fb5bff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:26.491999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ce4f3c63-4a45-461d-9ec4-63f024fb5bff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:26.596974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f3c63-4a45-461d-9ec4-63f024fb5bff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:26.598470 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4f3c63-4a45-461d-9ec4-63f024fb5bff service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:26.624277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1258/2499] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:27.029785 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb877ff-9513-4ba0-a605-5763a9099e09 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:27.041931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb877ff-9513-4ba0-a605-5763a9099e09 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:27.047240 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18139a-82ce-42fb-ab23-3812b2a0a207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:27.048246 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18139a-82ce-42fb-ab23-3812b2a0a207 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:27.048697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18139a-82ce-42fb-ab23-3812b2a0a207 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:27.049158 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18139a-82ce-42fb-ab23-3812b2a0a207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:27.051227 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7e18139a-82ce-42fb-ab23-3812b2a0a207 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:27.083803 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18139a-82ce-42fb-ab23-3812b2a0a207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:27.085429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18139a-82ce-42fb-ab23-3812b2a0a207 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:27.113374 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1242/2500] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:33.202745 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f3c63-4a45-461d-9ec4-63f024fb5bff service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:33.204241 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ce4f3c63-4a45-461d-9ec4-63f024fb5bff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:33.219346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4f3c63-4a45-461d-9ec4-63f024fb5bff service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:33.227029 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d25eb0-184d-4dac-ac1a-45220d5ccfb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:33.229056 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d25eb0-184d-4dac-ac1a-45220d5ccfb5 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:33.229584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d25eb0-184d-4dac-ac1a-45220d5ccfb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:33.229860 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d25eb0-184d-4dac-ac1a-45220d5ccfb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:33.231796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-50d25eb0-184d-4dac-ac1a-45220d5ccfb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:33.298956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:33.315666 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d25eb0-184d-4dac-ac1a-45220d5ccfb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:33.320364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d25eb0-184d-4dac-ac1a-45220d5ccfb5 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:33.330561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1259/2501] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:31:37.093069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=38 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:43.325402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=7 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:46.085657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b16bb-8736-40e2-a682-60a2dbaa9e29 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:46.086258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b16bb-8736-40e2-a682-60a2dbaa9e29 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:46.086372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96b16bb-8736-40e2-a682-60a2dbaa9e29 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:46.087769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1243/2502] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 15:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:46.094017 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac06dd-6317-4e45-8cb4-fa0bd9f2f8c0 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:46.094535 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac06dd-6317-4e45-8cb4-fa0bd9f2f8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:46.094654 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac06dd-6317-4e45-8cb4-fa0bd9f2f8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:46.116020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-93ac06dd-6317-4e45-8cb4-fa0bd9f2f8c0 None None] MFA Rules not processed for user `8a5fd31a2db944f596912e4f0aa26835`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:46.117279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-93ac06dd-6317-4e45-8cb4-fa0bd9f2f8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:46.127391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1260/2503] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 15:31:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:46.139391 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b16bb-8736-40e2-a682-60a2dbaa9e29 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:46.150833 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96b16bb-8736-40e2-a682-60a2dbaa9e29 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:46.154720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700df5db-815e-4710-a430-e33e1d5bd96f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '96c71ea88d9a474c9ff7a30555cfa91f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:46.155323 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700df5db-815e-4710-a430-e33e1d5bd96f service glance] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:46.155632 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700df5db-815e-4710-a430-e33e1d5bd96f service glance] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:46.155903 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700df5db-815e-4710-a430-e33e1d5bd96f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:46.169242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700df5db-815e-4710-a430-e33e1d5bd96f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:46.170702 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700df5db-815e-4710-a430-e33e1d5bd96f service glance] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:46.182492 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1244/2504] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:46.252171 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec541ab4-cb50-40ef-ad75-cdb60042bda1 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:46.252456 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec541ab4-cb50-40ef-ad75-cdb60042bda1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:46.252728 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec541ab4-cb50-40ef-ad75-cdb60042bda1 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:46.254089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1261/2505] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 15:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:46.259211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb47fb-00f1-46fc-942e-e3b76edcc2c2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:46.259527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb47fb-00f1-46fc-942e-e3b76edcc2c2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:46.259801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb47fb-00f1-46fc-942e-e3b76edcc2c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:46.279487 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c0eb47fb-00f1-46fc-942e-e3b76edcc2c2 None None] MFA Rules not processed for user `5427bd0ac88042338b956c10cfd07baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:46.280697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c0eb47fb-00f1-46fc-942e-e3b76edcc2c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:46.287474 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1245/2506] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 15:31:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:46.299593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec541ab4-cb50-40ef-ad75-cdb60042bda1 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:46.311445 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec541ab4-cb50-40ef-ad75-cdb60042bda1 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:46.315107 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bda575-16ca-4a4e-8dfb-0bd3b8d77b49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:46.315714 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bda575-16ca-4a4e-8dfb-0bd3b8d77b49 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:46.315995 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bda575-16ca-4a4e-8dfb-0bd3b8d77b49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:46.316267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bda575-16ca-4a4e-8dfb-0bd3b8d77b49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:46.324271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bda575-16ca-4a4e-8dfb-0bd3b8d77b49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:46.325702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bda575-16ca-4a4e-8dfb-0bd3b8d77b49 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:46.334023 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1262/2507] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:46.417907 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618247c4-170b-46a1-a683-400aec69c961 None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:46.418198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618247c4-170b-46a1-a683-400aec69c961 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:46.418507 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618247c4-170b-46a1-a683-400aec69c961 None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:46.419889 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1246/2508] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 15:31:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:31:46.424667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01692556-5007-4fd5-b9f1-6949efc276a9 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:46.424950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01692556-5007-4fd5-b9f1-6949efc276a9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:46.425222 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01692556-5007-4fd5-b9f1-6949efc276a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:46.447279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-01692556-5007-4fd5-b9f1-6949efc276a9 None None] MFA Rules not processed for user `8a5fd31a2db944f596912e4f0aa26835`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:46.448483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-01692556-5007-4fd5-b9f1-6949efc276a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:46.457669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1263/2509] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 15:31:46 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:46.468227 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618247c4-170b-46a1-a683-400aec69c961 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:46.479676 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618247c4-170b-46a1-a683-400aec69c961 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:46.483370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989e6f9-1cdf-4284-a901-87c12bca5a87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:46.483959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989e6f9-1cdf-4284-a901-87c12bca5a87 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:46.484232 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989e6f9-1cdf-4284-a901-87c12bca5a87 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:46.484515 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c989e6f9-1cdf-4284-a901-87c12bca5a87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:46.495068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989e6f9-1cdf-4284-a901-87c12bca5a87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:46.496486 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c989e6f9-1cdf-4284-a901-87c12bca5a87 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:46.507501 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1247/2510] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:46.570065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01692556-5007-4fd5-b9f1-6949efc276a9 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:46.571476 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-01692556-5007-4fd5-b9f1-6949efc276a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:46.647503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01692556-5007-4fd5-b9f1-6949efc276a9 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:46.655085 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b08985a-6e00-42df-8d0c-84d7f4abb896 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:46.655748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08985a-6e00-42df-8d0c-84d7f4abb896 service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:46.656056 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08985a-6e00-42df-8d0c-84d7f4abb896 service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:46.656358 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b08985a-6e00-42df-8d0c-84d7f4abb896 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:46.658558 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8b08985a-6e00-42df-8d0c-84d7f4abb896 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:46.671941 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08985a-6e00-42df-8d0c-84d7f4abb896 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:46.673546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b08985a-6e00-42df-8d0c-84d7f4abb896 service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:46.689416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1264/2511] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:56.502570 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=15 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:56.684277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=44 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:31:57.274599 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989e6f9-1cdf-4284-a901-87c12bca5a87 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:57.284129 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c989e6f9-1cdf-4284-a901-87c12bca5a87 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:57.288366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff304dc-cd1f-4e68-97d3-2c6ebf5dcef6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:57.288987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff304dc-cd1f-4e68-97d3-2c6ebf5dcef6 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:57.289265 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff304dc-cd1f-4e68-97d3-2c6ebf5dcef6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:57.289536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff304dc-cd1f-4e68-97d3-2c6ebf5dcef6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:57.291343 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2ff304dc-cd1f-4e68-97d3-2c6ebf5dcef6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:57.300970 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff304dc-cd1f-4e68-97d3-2c6ebf5dcef6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:57.302996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff304dc-cd1f-4e68-97d3-2c6ebf5dcef6 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:57.311195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1248/2512] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:31:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:31:58.134396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1111f5-5ffb-45b1-b9b6-1c62b04a3959 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:58.134879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1111f5-5ffb-45b1-b9b6-1c62b04a3959 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:58.134984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1111f5-5ffb-45b1-b9b6-1c62b04a3959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:58.154449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5f1111f5-5ffb-45b1-b9b6-1c62b04a3959 None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:31:58.155670 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5f1111f5-5ffb-45b1-b9b6-1c62b04a3959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:31:58.163124 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1265/2513] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 15:31:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:31:58.187762 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff304dc-cd1f-4e68-97d3-2c6ebf5dcef6 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:31:58.201801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff304dc-cd1f-4e68-97d3-2c6ebf5dcef6 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:31:58.207762 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3eb2a3-0589-46d4-b697-55d397b5ae10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:31:58.208477 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eb2a3-0589-46d4-b697-55d397b5ae10 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:31:58.208883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eb2a3-0589-46d4-b697-55d397b5ae10 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:31:58.209371 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3eb2a3-0589-46d4-b697-55d397b5ae10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:31:58.219382 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3eb2a3-0589-46d4-b697-55d397b5ae10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:31:58.221527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3eb2a3-0589-46d4-b697-55d397b5ae10 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:31:58.232461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1249/2514] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:31:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:32:08.148829 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:08.229353 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=12 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:19.433449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1111f5-5ffb-45b1-b9b6-1c62b04a3959 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:19.499856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1111f5-5ffb-45b1-b9b6-1c62b04a3959 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:19.506964 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111379fd-e5df-4517-ad94-e80785fe25d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:19.507612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111379fd-e5df-4517-ad94-e80785fe25d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:19.507895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111379fd-e5df-4517-ad94-e80785fe25d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:19.508163 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111379fd-e5df-4517-ad94-e80785fe25d9 admin admin] PATH_INFO: `/v3/users/88709c5fa47f4645a6aeda58cdec4ede` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:19.509422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111379fd-e5df-4517-ad94-e80785fe25d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88709c5fa47f4645a6aeda58cdec4ede)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:19.511651 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111379fd-e5df-4517-ad94-e80785fe25d9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:19.559679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-111379fd-e5df-4517-ad94-e80785fe25d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88709c5fa47f4645a6aeda58cdec4ede'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:19.567268 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-111379fd-e5df-4517-ad94-e80785fe25d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88709c5fa47f4645a6aeda58cdec4ede'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:19.580974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-111379fd-e5df-4517-ad94-e80785fe25d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88709c5fa47f4645a6aeda58cdec4ede'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:19.581738 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-111379fd-e5df-4517-ad94-e80785fe25d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88709c5fa47f4645a6aeda58cdec4ede'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:19.589176 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-111379fd-e5df-4517-ad94-e80785fe25d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88709c5fa47f4645a6aeda58cdec4ede'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:19.595240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1266/2515] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:32:19 2022] DELETE /identity/v3/users/88709c5fa47f4645a6aeda58cdec4ede => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:32:19.600195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746eec89-47df-4004-a587-c5cdcb6b306f None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:19.600706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746eec89-47df-4004-a587-c5cdcb6b306f None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:19.600706 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746eec89-47df-4004-a587-c5cdcb6b306f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:19.620585 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-746eec89-47df-4004-a587-c5cdcb6b306f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:32:19.621810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-746eec89-47df-4004-a587-c5cdcb6b306f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:19.681453 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1250/2516] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:32:19.691273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111379fd-e5df-4517-ad94-e80785fe25d9 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:19.692683 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-111379fd-e5df-4517-ad94-e80785fe25d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:19.748884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111379fd-e5df-4517-ad94-e80785fe25d9 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:19.753036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7e365-7660-4916-9ac0-e8b742502ed5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:19.754022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7e365-7660-4916-9ac0-e8b742502ed5 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:19.754418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7e365-7660-4916-9ac0-e8b742502ed5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:19.754940 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb7e365-7660-4916-9ac0-e8b742502ed5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:19.764990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7e365-7660-4916-9ac0-e8b742502ed5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:19.767217 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb7e365-7660-4916-9ac0-e8b742502ed5 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:19.776479 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1267/2517] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:32:20.058498 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746eec89-47df-4004-a587-c5cdcb6b306f None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:20.060170 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-746eec89-47df-4004-a587-c5cdcb6b306f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:20.077264 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746eec89-47df-4004-a587-c5cdcb6b306f None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:20.081817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:20.083010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:20.083010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:20.083010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90 admin admin] PATH_INFO: `/v3/projects/a17f14aab8fb4fc4b2240906b314a865` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:20.084173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a17f14aab8fb4fc4b2240906b314a865)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:20.086088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:20.138422 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a17f14aab8fb4fc4b2240906b314a865', 'request_id': 'req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:20.147525 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a17f14aab8fb4fc4b2240906b314a865', 'request_id': 'req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:20.161355 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a17f14aab8fb4fc4b2240906b314a865', 'request_id': 'req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:20.173541 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90 admin admin] The token cache is being invalidate because project a17f14aab8fb4fc4b2240906b314a865 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:32:20.173901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:20.174649 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:20.176743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1251/2518] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:32:20 2022] DELETE /identity/v3/projects/a17f14aab8fb4fc4b2240906b314a865 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:32:27.559702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7e365-7660-4916-9ac0-e8b742502ed5 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:27.561152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cdb7e365-7660-4916-9ac0-e8b742502ed5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:27.589598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:27.631433 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:27.638654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:27.656634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb7e365-7660-4916-9ac0-e8b742502ed5 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:27.736016 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f42c9-fb8a-4aa5-925a-7d4352db5c90 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:27.739378 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43db59-419d-4c30-98d0-63e2926db85e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:27.740020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43db59-419d-4c30-98d0-63e2926db85e service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:27.740397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43db59-419d-4c30-98d0-63e2926db85e service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:27.740921 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c43db59-419d-4c30-98d0-63e2926db85e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:27.743101 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c005fe-0cc6-46bc-ac69-8b8bdff1ea30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:27.743660 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c005fe-0cc6-46bc-ac69-8b8bdff1ea30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:27.743847 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-1c43db59-419d-4c30-98d0-63e2926db85e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:27.744192 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c005fe-0cc6-46bc-ac69-8b8bdff1ea30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:27.744533 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c005fe-0cc6-46bc-ac69-8b8bdff1ea30 admin admin] PATH_INFO: `/v3/users/22c5f040350d4daba0ebe2af86849a23` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:27.745920 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c005fe-0cc6-46bc-ac69-8b8bdff1ea30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22c5f040350d4daba0ebe2af86849a23)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:27.748182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c005fe-0cc6-46bc-ac69-8b8bdff1ea30 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:27.757834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43db59-419d-4c30-98d0-63e2926db85e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:27.759536 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c43db59-419d-4c30-98d0-63e2926db85e service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:27.773782 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1268/2519] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:32:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:32:27.788813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:27.823056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-75c005fe-0cc6-46bc-ac69-8b8bdff1ea30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22c5f040350d4daba0ebe2af86849a23'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:27.823873 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-75c005fe-0cc6-46bc-ac69-8b8bdff1ea30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22c5f040350d4daba0ebe2af86849a23'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:27.837444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-75c005fe-0cc6-46bc-ac69-8b8bdff1ea30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22c5f040350d4daba0ebe2af86849a23'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:27.846054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-75c005fe-0cc6-46bc-ac69-8b8bdff1ea30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22c5f040350d4daba0ebe2af86849a23'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:27.846684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-75c005fe-0cc6-46bc-ac69-8b8bdff1ea30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22c5f040350d4daba0ebe2af86849a23'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:27.859695 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1252/2520] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:32:27 2022] DELETE /identity/v3/users/22c5f040350d4daba0ebe2af86849a23 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:32:28.047880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43db59-419d-4c30-98d0-63e2926db85e service placement] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:28.060794 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c43db59-419d-4c30-98d0-63e2926db85e service placement] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:28.103473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c005fe-0cc6-46bc-ac69-8b8bdff1ea30 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:28.105716 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-75c005fe-0cc6-46bc-ac69-8b8bdff1ea30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:28.133458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8429be76-ec79-43da-894e-b415136d49d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:28.133458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429be76-ec79-43da-894e-b415136d49d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:28.133458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429be76-ec79-43da-894e-b415136d49d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:28.133458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8429be76-ec79-43da-894e-b415136d49d3 admin admin] PATH_INFO: `/v3/projects/00071111c84346639e5245e5c173dad5` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:28.134527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429be76-ec79-43da-894e-b415136d49d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00071111c84346639e5245e5c173dad5)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:28.136884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8429be76-ec79-43da-894e-b415136d49d3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:28.200127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8429be76-ec79-43da-894e-b415136d49d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00071111c84346639e5245e5c173dad5', 'request_id': 'req-8429be76-ec79-43da-894e-b415136d49d3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:28.207443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8429be76-ec79-43da-894e-b415136d49d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00071111c84346639e5245e5c173dad5', 'request_id': 'req-8429be76-ec79-43da-894e-b415136d49d3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:28.207838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c005fe-0cc6-46bc-ac69-8b8bdff1ea30 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:28.210390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:28.218199 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8429be76-ec79-43da-894e-b415136d49d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00071111c84346639e5245e5c173dad5', 'request_id': 'req-8429be76-ec79-43da-894e-b415136d49d3'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:28.234857 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8429be76-ec79-43da-894e-b415136d49d3 admin admin] The token cache is being invalidate because project 00071111c84346639e5245e5c173dad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:32:28.235337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8429be76-ec79-43da-894e-b415136d49d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:28.236411 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-8429be76-ec79-43da-894e-b415136d49d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:32:28.239584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1269/2521] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:32:28 2022] DELETE /identity/v3/projects/00071111c84346639e5245e5c173dad5 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:32:28.249668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea124b95-1777-4215-864b-fc7d158e1d3f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:28.250067 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea124b95-1777-4215-864b-fc7d158e1d3f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:28.250373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea124b95-1777-4215-864b-fc7d158e1d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:28.272093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ea124b95-1777-4215-864b-fc7d158e1d3f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:32:28.273314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ea124b95-1777-4215-864b-fc7d158e1d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:28.286174 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ff3594-ebfc-48f9-a7e5-ad72b19f089d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:28.286813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff3594-ebfc-48f9-a7e5-ad72b19f089d service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:28.287087 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff3594-ebfc-48f9-a7e5-ad72b19f089d service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:28.287354 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ff3594-ebfc-48f9-a7e5-ad72b19f089d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:28.289107 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f0ff3594-ebfc-48f9-a7e5-ad72b19f089d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:28.301413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff3594-ebfc-48f9-a7e5-ad72b19f089d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:28.303065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ff3594-ebfc-48f9-a7e5-ad72b19f089d service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:28.315294 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1253/2522] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:32:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:32:28.338987 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1270/2523] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:32:28.347004 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd02b7c-b9fb-46e1-b847-1559a77255c7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:28.347292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd02b7c-b9fb-46e1-b847-1559a77255c7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:28.348145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd02b7c-b9fb-46e1-b847-1559a77255c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:28.372776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1cd02b7c-b9fb-46e1-b847-1559a77255c7 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:32:28.373994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1cd02b7c-b9fb-46e1-b847-1559a77255c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:28.382291 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1254/2524] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:32:28.387540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea124b95-1777-4215-864b-fc7d158e1d3f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:28.398159 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea124b95-1777-4215-864b-fc7d158e1d3f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:28.402999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0aab21-0786-4184-ac48-4b869964fcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:28.403598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0aab21-0786-4184-ac48-4b869964fcd2 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:28.403872 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0aab21-0786-4184-ac48-4b869964fcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:28.404137 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0aab21-0786-4184-ac48-4b869964fcd2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:28.405141 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0aab21-0786-4184-ac48-4b869964fcd2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:32:28.405440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0aab21-0786-4184-ac48-4b869964fcd2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:28.407469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0aab21-0786-4184-ac48-4b869964fcd2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:28.420778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1271/2525] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:32:28.425068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd02b7c-b9fb-46e1-b847-1559a77255c7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:28.434505 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd02b7c-b9fb-46e1-b847-1559a77255c7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:28.439480 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aff224-dd0a-46ae-b8d1-de23a84728d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:28.440080 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aff224-dd0a-46ae-b8d1-de23a84728d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:28.440357 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aff224-dd0a-46ae-b8d1-de23a84728d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:28.440627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0aff224-dd0a-46ae-b8d1-de23a84728d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:28.441764 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aff224-dd0a-46ae-b8d1-de23a84728d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:28.443678 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0aff224-dd0a-46ae-b8d1-de23a84728d0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:28.465792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:28.474260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1255/2526] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:32:28 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:32:28.478036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0aab21-0786-4184-ac48-4b869964fcd2 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:28.488210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0aab21-0786-4184-ac48-4b869964fcd2 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:28.559259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a56e73-5156-4e9e-b525-33eb5473f909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:28.559902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a56e73-5156-4e9e-b525-33eb5473f909 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:28.560440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a56e73-5156-4e9e-b525-33eb5473f909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:28.560440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a56e73-5156-4e9e-b525-33eb5473f909 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:28.561067 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a56e73-5156-4e9e-b525-33eb5473f909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:28.563242 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a56e73-5156-4e9e-b525-33eb5473f909 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:28.588651 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1272/2527] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:32:28 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:32:28.593039 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aff224-dd0a-46ae-b8d1-de23a84728d0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:28.605767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0aff224-dd0a-46ae-b8d1-de23a84728d0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:28.610677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861e055-9912-4c2a-a18c-52420353470a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:28.611289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861e055-9912-4c2a-a18c-52420353470a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:28.611568 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861e055-9912-4c2a-a18c-52420353470a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:28.611848 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f861e055-9912-4c2a-a18c-52420353470a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:28.612489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861e055-9912-4c2a-a18c-52420353470a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:28.614117 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f861e055-9912-4c2a-a18c-52420353470a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:28.627093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1256/2528] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:32:28.631886 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a56e73-5156-4e9e-b525-33eb5473f909 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:28.642320 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a56e73-5156-4e9e-b525-33eb5473f909 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:28.647545 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab3939e-d8f5-4400-bff3-e6fc9ff30c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:28.648125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3939e-d8f5-4400-bff3-e6fc9ff30c21 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:28.648396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3939e-d8f5-4400-bff3-e6fc9ff30c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:28.648662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab3939e-d8f5-4400-bff3-e6fc9ff30c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:28.649289 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3939e-d8f5-4400-bff3-e6fc9ff30c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:28.651322 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab3939e-d8f5-4400-bff3-e6fc9ff30c21 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:28.663130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1273/2529] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:32:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:32:28.667502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861e055-9912-4c2a-a18c-52420353470a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:28.678524 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f861e055-9912-4c2a-a18c-52420353470a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:28.683671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd2220-8ece-4016-a744-fc129271a9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:28.684368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd2220-8ece-4016-a744-fc129271a9bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:28.684772 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd2220-8ece-4016-a744-fc129271a9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:28.685082 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dd2220-8ece-4016-a744-fc129271a9bf admin admin] PATH_INFO: `/v3/projects/68476a44d7ee4d1d9fbf3c7516885edf/users/63b945e0d46946bfad2d89a0fd15fd8c/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:28.700821 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd2220-8ece-4016-a744-fc129271a9bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=68476a44d7ee4d1d9fbf3c7516885edf, user_id=63b945e0d46946bfad2d89a0fd15fd8c, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:28.703484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09dd2220-8ece-4016-a744-fc129271a9bf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:28.717784 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1257/2530] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:32:28 2022] PUT /identity/v3/projects/68476a44d7ee4d1d9fbf3c7516885edf/users/63b945e0d46946bfad2d89a0fd15fd8c/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:32:28.723221 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8228e0d-8b2b-43cb-830e-d6074d7659cd None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:28.723509 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8228e0d-8b2b-43cb-830e-d6074d7659cd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:28.723791 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8228e0d-8b2b-43cb-830e-d6074d7659cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:28.743449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c8228e0d-8b2b-43cb-830e-d6074d7659cd None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:32:28.744706 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c8228e0d-8b2b-43cb-830e-d6074d7659cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:28.797614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:28.815158 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1274/2531] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:32:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:32:28.828258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd2220-8ece-4016-a744-fc129271a9bf admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:28.830040 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-09dd2220-8ece-4016-a744-fc129271a9bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:28.900760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dd2220-8ece-4016-a744-fc129271a9bf admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:28.904793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae8f3c-f8af-4233-abbd-aa92080f2266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:28.905404 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae8f3c-f8af-4233-abbd-aa92080f2266 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:28.905679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae8f3c-f8af-4233-abbd-aa92080f2266 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:28.905946 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae8f3c-f8af-4233-abbd-aa92080f2266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:28.915543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae8f3c-f8af-4233-abbd-aa92080f2266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:28.917121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae8f3c-f8af-4233-abbd-aa92080f2266 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:28.927434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1258/2532] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:32:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:32:31.143024 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2929d-80e5-4191-94d7-fc7923e76951 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:31.143562 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2929d-80e5-4191-94d7-fc7923e76951 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:31.143562 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb2929d-80e5-4191-94d7-fc7923e76951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:31.170116 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cfb2929d-80e5-4191-94d7-fc7923e76951 None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:32:31.170893 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cfb2929d-80e5-4191-94d7-fc7923e76951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:31.242768 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1275/2533] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 24 15:32:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:32:31.259486 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae8f3c-f8af-4233-abbd-aa92080f2266 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:31.263296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c1ae8f3c-f8af-4233-abbd-aa92080f2266 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:31.358536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae8f3c-f8af-4233-abbd-aa92080f2266 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:31.370536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4ea6e-daee-455a-ba6f-a3f593a97cfb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:31.371280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4ea6e-daee-455a-ba6f-a3f593a97cfb service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:31.374017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4ea6e-daee-455a-ba6f-a3f593a97cfb service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:31.374017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d4ea6e-daee-455a-ba6f-a3f593a97cfb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:31.382068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4ea6e-daee-455a-ba6f-a3f593a97cfb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:31.384366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d4ea6e-daee-455a-ba6f-a3f593a97cfb service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:31.395035 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1259/2534] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:32:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:32:33.530388 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d73f86-6869-4f8f-829e-8453fb82577c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:33.530880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d73f86-6869-4f8f-829e-8453fb82577c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:33.530979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d73f86-6869-4f8f-829e-8453fb82577c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:33.561864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d9d73f86-6869-4f8f-829e-8453fb82577c None None] MFA Rules not processed for user `63b945e0d46946bfad2d89a0fd15fd8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:32:33.563385 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d9d73f86-6869-4f8f-829e-8453fb82577c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:33.636911 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1276/2535] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:32:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:32:33.642388 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4ea6e-daee-455a-ba6f-a3f593a97cfb service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:33.659542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d4ea6e-daee-455a-ba6f-a3f593a97cfb service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:33.664307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e055aec-dc2a-44cb-90e5-df11b4cbba59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:33.665007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e055aec-dc2a-44cb-90e5-df11b4cbba59 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:33.665446 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e055aec-dc2a-44cb-90e5-df11b4cbba59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:33.665799 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e055aec-dc2a-44cb-90e5-df11b4cbba59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:33.666552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e055aec-dc2a-44cb-90e5-df11b4cbba59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:33.668507 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e055aec-dc2a-44cb-90e5-df11b4cbba59 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:33.704814 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1260/2536] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:32:33 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:32:33.706998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d73f86-6869-4f8f-829e-8453fb82577c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:33.719143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d73f86-6869-4f8f-829e-8453fb82577c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:33.786957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862306e7-7a5a-4e78-acf8-041351c45d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:33.787604 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862306e7-7a5a-4e78-acf8-041351c45d18 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:33.787904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862306e7-7a5a-4e78-acf8-041351c45d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:33.788176 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862306e7-7a5a-4e78-acf8-041351c45d18 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:33.788817 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862306e7-7a5a-4e78-acf8-041351c45d18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:33.790903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862306e7-7a5a-4e78-acf8-041351c45d18 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:33.815025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1277/2537] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:32:33 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:32:33.819155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e055aec-dc2a-44cb-90e5-df11b4cbba59 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:33.835050 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e055aec-dc2a-44cb-90e5-df11b4cbba59 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:33.840883 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc734219-98ba-4557-a07d-1797da04f406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:33.841592 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc734219-98ba-4557-a07d-1797da04f406 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:33.841947 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc734219-98ba-4557-a07d-1797da04f406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:33.842335 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc734219-98ba-4557-a07d-1797da04f406 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:33.843067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc734219-98ba-4557-a07d-1797da04f406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:33.845129 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc734219-98ba-4557-a07d-1797da04f406 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:33.857900 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1261/2538] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:32:33.864086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862306e7-7a5a-4e78-acf8-041351c45d18 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:33.874816 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862306e7-7a5a-4e78-acf8-041351c45d18 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:33.879508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e123d7a-634c-4363-a6cd-9a0924b2077f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:33.880150 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e123d7a-634c-4363-a6cd-9a0924b2077f admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:33.880428 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e123d7a-634c-4363-a6cd-9a0924b2077f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:33.880702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e123d7a-634c-4363-a6cd-9a0924b2077f admin admin] PATH_INFO: `/v3/projects/5939401400f5466ba706e2df4f20c72f/users/88271ab2d084420ab6d57f5d63c0b12f/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:33.893528 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:33.897524 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e123d7a-634c-4363-a6cd-9a0924b2077f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5939401400f5466ba706e2df4f20c72f, user_id=88271ab2d084420ab6d57f5d63c0b12f, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:33.901107 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e123d7a-634c-4363-a6cd-9a0924b2077f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:33.916888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1278/2539] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:32:33 2022] PUT /identity/v3/projects/5939401400f5466ba706e2df4f20c72f/users/88271ab2d084420ab6d57f5d63c0b12f/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:32:33.920695 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc734219-98ba-4557-a07d-1797da04f406 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:33.933338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc734219-98ba-4557-a07d-1797da04f406 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:33.960959 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:34.003490 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268028aa-befc-4a8d-ad94-05312dffa889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:34.004113 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268028aa-befc-4a8d-ad94-05312dffa889 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:34.004424 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268028aa-befc-4a8d-ad94-05312dffa889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:34.004665 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268028aa-befc-4a8d-ad94-05312dffa889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:34.005314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268028aa-befc-4a8d-ad94-05312dffa889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:34.006955 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268028aa-befc-4a8d-ad94-05312dffa889 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:34.018590 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1262/2540] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:32:34.022757 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e123d7a-634c-4363-a6cd-9a0924b2077f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:34.038306 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e123d7a-634c-4363-a6cd-9a0924b2077f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:34.047022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e759b8-0bb3-4460-ad2c-3a0eb1605a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:34.047775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e759b8-0bb3-4460-ad2c-3a0eb1605a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:34.048157 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e759b8-0bb3-4460-ad2c-3a0eb1605a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:34.048499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e759b8-0bb3-4460-ad2c-3a0eb1605a3c admin admin] PATH_INFO: `/v3/projects/5939401400f5466ba706e2df4f20c72f/users/88271ab2d084420ab6d57f5d63c0b12f/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:34.051437 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e759b8-0bb3-4460-ad2c-3a0eb1605a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5939401400f5466ba706e2df4f20c72f, user_id=88271ab2d084420ab6d57f5d63c0b12f, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:34.060497 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e759b8-0bb3-4460-ad2c-3a0eb1605a3c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:34.071818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1279/2541] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:32:34 2022] PUT /identity/v3/projects/5939401400f5466ba706e2df4f20c72f/users/88271ab2d084420ab6d57f5d63c0b12f/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:32:38.221005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3329e-b3d7-468d-b73d-38976368372c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:38.221527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3329e-b3d7-468d-b73d-38976368372c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:38.221527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3329e-b3d7-468d-b73d-38976368372c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:38.252156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fca3329e-b3d7-468d-b73d-38976368372c None None] MFA Rules not processed for user `88271ab2d084420ab6d57f5d63c0b12f`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:32:38.254341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fca3329e-b3d7-468d-b73d-38976368372c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:38.338406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1263/2542] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:32:38 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:32:38.348183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e759b8-0bb3-4460-ad2c-3a0eb1605a3c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:32:38.349982 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d6e759b8-0bb3-4460-ad2c-3a0eb1605a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:38.459485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e759b8-0bb3-4460-ad2c-3a0eb1605a3c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:32:38.468090 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302d26f-7724-48c0-9e47-903fbe0452c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:32:38.468963 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302d26f-7724-48c0-9e47-903fbe0452c7 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:38.469183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302d26f-7724-48c0-9e47-903fbe0452c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:38.469460 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8302d26f-7724-48c0-9e47-903fbe0452c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:38.544352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302d26f-7724-48c0-9e47-903fbe0452c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:32:38.545990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8302d26f-7724-48c0-9e47-903fbe0452c7 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:32:38.562601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1280/2543] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:32:38 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:32:39.785491 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2324e4-6c50-41aa-be16-c6d17ba08319 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:32:39.786036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2324e4-6c50-41aa-be16-c6d17ba08319 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:32:39.786479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2324e4-6c50-41aa-be16-c6d17ba08319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:32:39.808739 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4e2324e4-6c50-41aa-be16-c6d17ba08319 None None] MFA Rules not processed for user `63b945e0d46946bfad2d89a0fd15fd8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:32:39.810136 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4e2324e4-6c50-41aa-be16-c6d17ba08319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:32:39.819245 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1264/2544] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:32:39 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:32:48.552426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=64,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:32:49.802417 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=56 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:33:22.277036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302d26f-7724-48c0-9e47-903fbe0452c7 service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:33:22.278512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8302d26f-7724-48c0-9e47-903fbe0452c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:22.336463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8302d26f-7724-48c0-9e47-903fbe0452c7 service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:33:22.340081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69454045-37d1-4a36-9d3d-67b53a24f3b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:33:22.340680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69454045-37d1-4a36-9d3d-67b53a24f3b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:22.340957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69454045-37d1-4a36-9d3d-67b53a24f3b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:22.341227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69454045-37d1-4a36-9d3d-67b53a24f3b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:22.351068 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69454045-37d1-4a36-9d3d-67b53a24f3b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:33:22.352507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69454045-37d1-4a36-9d3d-67b53a24f3b4 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:33:22.362580 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1281/2545] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:33:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:33:32.357880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=26 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:33:35.755303 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2324e4-6c50-41aa-be16-c6d17ba08319 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:33:35.768346 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2324e4-6c50-41aa-be16-c6d17ba08319 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:33:35.773803 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d481f43a-f91d-4297-9e94-a800b67d08c6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:33:35.774435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481f43a-f91d-4297-9e94-a800b67d08c6 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:33:35.774712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481f43a-f91d-4297-9e94-a800b67d08c6 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:33:35.774985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d481f43a-f91d-4297-9e94-a800b67d08c6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:33:35.776690 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d481f43a-f91d-4297-9e94-a800b67d08c6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:33:35.838530 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481f43a-f91d-4297-9e94-a800b67d08c6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:33:35.840064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d481f43a-f91d-4297-9e94-a800b67d08c6 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:33:35.850603 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1265/2546] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:33:35 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:33:45.845461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=28 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:34:00.564993 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4a654-fcf8-4266-955f-cee67a754895 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:34:00.565568 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4a654-fcf8-4266-955f-cee67a754895 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:34:00.565568 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b4a654-fcf8-4266-955f-cee67a754895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:34:00.585063 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-91b4a654-fcf8-4266-955f-cee67a754895 None None] MFA Rules not processed for user `461b9a3987744067a62234aaa8d086d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:34:00.586293 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-91b4a654-fcf8-4266-955f-cee67a754895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:34:00.593458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1282/2547] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:34:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:34:00.835286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6c141-0dc9-4f1d-a639-96cc20698305 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:34:00.835746 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6c141-0dc9-4f1d-a639-96cc20698305 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:34:00.835848 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6c141-0dc9-4f1d-a639-96cc20698305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:34:00.855974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-abf6c141-0dc9-4f1d-a639-96cc20698305 None None] MFA Rules not processed for user `56535a702f4344aa8a58f11406acf4c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:34:00.857182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-abf6c141-0dc9-4f1d-a639-96cc20698305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:34:00.918924 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1266/2548] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:34:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:34:01.413264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4a654-fcf8-4266-955f-cee67a754895 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:34:01.423390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b4a654-fcf8-4266-955f-cee67a754895 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:34:01.426958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a52e96-4df5-40d4-bbed-bbe887bed918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:34:01.427564 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a52e96-4df5-40d4-bbed-bbe887bed918 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:34:01.427842 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a52e96-4df5-40d4-bbed-bbe887bed918 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:34:01.428110 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a52e96-4df5-40d4-bbed-bbe887bed918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:34:01.429850 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a8a52e96-4df5-40d4-bbed-bbe887bed918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:34:01.440432 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a52e96-4df5-40d4-bbed-bbe887bed918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:34:01.441874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a52e96-4df5-40d4-bbed-bbe887bed918 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:34:01.450077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1283/2549] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:34:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:34:10.911514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=25 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:34:11.446531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=9 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:34:41.699932 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6c141-0dc9-4f1d-a639-96cc20698305 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:34:41.701338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-abf6c141-0dc9-4f1d-a639-96cc20698305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:34:41.841271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6c141-0dc9-4f1d-a639-96cc20698305 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:34:41.846338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0aaaa-ac01-4944-a00d-10d305249c9c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:34:41.846972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0aaaa-ac01-4944-a00d-10d305249c9c service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:34:41.847335 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0aaaa-ac01-4944-a00d-10d305249c9c service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:34:41.847613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a0aaaa-ac01-4944-a00d-10d305249c9c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:34:41.849345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-77a0aaaa-ac01-4944-a00d-10d305249c9c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:34:41.861042 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a0aaaa-ac01-4944-a00d-10d305249c9c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:34:41.862640 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a0aaaa-ac01-4944-a00d-10d305249c9c service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:34:41.872424 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1267/2550] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:34:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:34:51.868388 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=63 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:34:57.511822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a52e96-4df5-40d4-bbed-bbe887bed918 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:34:57.513253 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a8a52e96-4df5-40d4-bbed-bbe887bed918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:34:57.585961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a52e96-4df5-40d4-bbed-bbe887bed918 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:34:57.591474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f8beb7-fb01-404c-96e7-7022499a6220 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:34:57.592065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8beb7-fb01-404c-96e7-7022499a6220 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:34:57.592385 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8beb7-fb01-404c-96e7-7022499a6220 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:34:57.592656 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f8beb7-fb01-404c-96e7-7022499a6220 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:34:57.594400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f2f8beb7-fb01-404c-96e7-7022499a6220 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:34:57.652066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8beb7-fb01-404c-96e7-7022499a6220 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:34:57.653719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f8beb7-fb01-404c-96e7-7022499a6220 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:34:57.663414 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1284/2551] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:34:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:34:58.133306 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0aaaa-ac01-4944-a00d-10d305249c9c service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:34:58.149599 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a0aaaa-ac01-4944-a00d-10d305249c9c service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:34:58.158106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918cb49-c694-421b-bbd4-0aed50b4fd4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:34:58.159102 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918cb49-c694-421b-bbd4-0aed50b4fd4e service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:34:58.159774 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918cb49-c694-421b-bbd4-0aed50b4fd4e service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:34:58.160308 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1918cb49-c694-421b-bbd4-0aed50b4fd4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:34:58.162287 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1918cb49-c694-421b-bbd4-0aed50b4fd4e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:34:58.179695 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918cb49-c694-421b-bbd4-0aed50b4fd4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:34:58.182008 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1918cb49-c694-421b-bbd4-0aed50b4fd4e service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:34:58.195597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1268/2552] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:34:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:05.548281 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9355b-8ba1-48fb-89f6-294e39ffe7bd None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:05.548767 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9355b-8ba1-48fb-89f6-294e39ffe7bd None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:05.548869 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9355b-8ba1-48fb-89f6-294e39ffe7bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:05.579311 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-45e9355b-8ba1-48fb-89f6-294e39ffe7bd None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:05.580439 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-45e9355b-8ba1-48fb-89f6-294e39ffe7bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:05.634881 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1285/2553] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 24 15:35:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:05.680853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918cb49-c694-421b-bbd4-0aed50b4fd4e service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:05.746213 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1918cb49-c694-421b-bbd4-0aed50b4fd4e service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:05.751192 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d0477-b7bc-4b94-b044-20d56ede5f92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:05.751812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d0477-b7bc-4b94-b044-20d56ede5f92 service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:05.752090 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d0477-b7bc-4b94-b044-20d56ede5f92 service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:05.752363 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29d0477-b7bc-4b94-b044-20d56ede5f92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:05.760533 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d0477-b7bc-4b94-b044-20d56ede5f92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:05.762151 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29d0477-b7bc-4b94-b044-20d56ede5f92 service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:05.770607 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1269/2554] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:35:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:35:06.113318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9355b-8ba1-48fb-89f6-294e39ffe7bd None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:06.132152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9355b-8ba1-48fb-89f6-294e39ffe7bd None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:06.135716 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c1fb3-0bce-4529-84bb-3c22140f31d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:06.136310 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c1fb3-0bce-4529-84bb-3c22140f31d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:06.136617 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c1fb3-0bce-4529-84bb-3c22140f31d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:06.136894 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c1fb3-0bce-4529-84bb-3c22140f31d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:06.138651 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4a0c1fb3-0bce-4529-84bb-3c22140f31d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:06.170015 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:06.224813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c1fb3-0bce-4529-84bb-3c22140f31d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:06.226270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c1fb3-0bce-4529-84bb-3c22140f31d4 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:06.245350 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1286/2555] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:12.648438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d0477-b7bc-4b94-b044-20d56ede5f92 service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:12.666111 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29d0477-b7bc-4b94-b044-20d56ede5f92 service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:12.673061 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51ac91-8b50-4fad-a909-101d406b2405 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:12.673808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51ac91-8b50-4fad-a909-101d406b2405 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:12.674158 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51ac91-8b50-4fad-a909-101d406b2405 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:12.674682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d51ac91-8b50-4fad-a909-101d406b2405 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:12.676564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0d51ac91-8b50-4fad-a909-101d406b2405 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:12.690473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51ac91-8b50-4fad-a909-101d406b2405 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:12.692254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d51ac91-8b50-4fad-a909-101d406b2405 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:12.705855 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1270/2556] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:12.721612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c1fb3-0bce-4529-84bb-3c22140f31d4 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:12.723850 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-4a0c1fb3-0bce-4529-84bb-3c22140f31d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:12.740382 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c1fb3-0bce-4529-84bb-3c22140f31d4 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:12.746068 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c3bda-33ac-435d-b405-e3cd420188b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:12.747095 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c3bda-33ac-435d-b405-e3cd420188b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:12.747670 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c3bda-33ac-435d-b405-e3cd420188b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:12.748014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c3bda-33ac-435d-b405-e3cd420188b4 admin admin] PATH_INFO: `/v3/users/24396d51e9d243f7920f9fc309d66115` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:12.749440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c3bda-33ac-435d-b405-e3cd420188b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24396d51e9d243f7920f9fc309d66115)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:12.752324 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c3bda-33ac-435d-b405-e3cd420188b4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:12.804568 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-249c3bda-33ac-435d-b405-e3cd420188b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24396d51e9d243f7920f9fc309d66115'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:12.811706 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-249c3bda-33ac-435d-b405-e3cd420188b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24396d51e9d243f7920f9fc309d66115'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:12.824036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-249c3bda-33ac-435d-b405-e3cd420188b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24396d51e9d243f7920f9fc309d66115'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:12.824687 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-249c3bda-33ac-435d-b405-e3cd420188b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24396d51e9d243f7920f9fc309d66115'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:12.831567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-249c3bda-33ac-435d-b405-e3cd420188b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24396d51e9d243f7920f9fc309d66115'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:12.837753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1287/2557] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:35:12 2022] DELETE /identity/v3/users/24396d51e9d243f7920f9fc309d66115 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:13.017254 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51ac91-8b50-4fad-a909-101d406b2405 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:13.031042 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d51ac91-8b50-4fad-a909-101d406b2405 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:13.090940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacd4ae-51c2-4d71-913c-cf226b174722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:13.091484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacd4ae-51c2-4d71-913c-cf226b174722 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:13.091760 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacd4ae-51c2-4d71-913c-cf226b174722 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:13.092607 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebacd4ae-51c2-4d71-913c-cf226b174722 admin admin] PATH_INFO: `/v3/projects/60386a4f71754e4396434dfa34c55b75` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:13.093723 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebacd4ae-51c2-4d71-913c-cf226b174722 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60386a4f71754e4396434dfa34c55b75)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:13.095962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebacd4ae-51c2-4d71-913c-cf226b174722 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:13.144624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ebacd4ae-51c2-4d71-913c-cf226b174722 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60386a4f71754e4396434dfa34c55b75', 'request_id': 'req-ebacd4ae-51c2-4d71-913c-cf226b174722'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:13.149985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ebacd4ae-51c2-4d71-913c-cf226b174722 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60386a4f71754e4396434dfa34c55b75', 'request_id': 'req-ebacd4ae-51c2-4d71-913c-cf226b174722'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:13.157152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ebacd4ae-51c2-4d71-913c-cf226b174722 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60386a4f71754e4396434dfa34c55b75', 'request_id': 'req-ebacd4ae-51c2-4d71-913c-cf226b174722'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:13.158677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:13.172432 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ebacd4ae-51c2-4d71-913c-cf226b174722 admin admin] The token cache is being invalidate because project 60386a4f71754e4396434dfa34c55b75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:35:13.172719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ebacd4ae-51c2-4d71-913c-cf226b174722 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:13.173351 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ebacd4ae-51c2-4d71-913c-cf226b174722 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:13.174873 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1271/2558] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:35:13 2022] DELETE /identity/v3/projects/60386a4f71754e4396434dfa34c55b75 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:13.179085 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c3bda-33ac-435d-b405-e3cd420188b4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:13.180492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-249c3bda-33ac-435d-b405-e3cd420188b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:13.193668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c3bda-33ac-435d-b405-e3cd420188b4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:13.248978 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4031825-bef7-430c-9e4e-89332906eb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:13.249639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4031825-bef7-430c-9e4e-89332906eb8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:13.249915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4031825-bef7-430c-9e4e-89332906eb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:13.250187 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4031825-bef7-430c-9e4e-89332906eb8f admin admin] PATH_INFO: `/v3/users/ca78e1925cd54b679ddd1406aaf4d9b2` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:13.251336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4031825-bef7-430c-9e4e-89332906eb8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca78e1925cd54b679ddd1406aaf4d9b2)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:13.253264 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4031825-bef7-430c-9e4e-89332906eb8f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:13.263024 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:13.301206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a4031825-bef7-430c-9e4e-89332906eb8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca78e1925cd54b679ddd1406aaf4d9b2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:13.308595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a4031825-bef7-430c-9e4e-89332906eb8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca78e1925cd54b679ddd1406aaf4d9b2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:13.320312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a4031825-bef7-430c-9e4e-89332906eb8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca78e1925cd54b679ddd1406aaf4d9b2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:13.320950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a4031825-bef7-430c-9e4e-89332906eb8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca78e1925cd54b679ddd1406aaf4d9b2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:13.328185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a4031825-bef7-430c-9e4e-89332906eb8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca78e1925cd54b679ddd1406aaf4d9b2'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:13.333862 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1288/2559] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:35:13 2022] DELETE /identity/v3/users/ca78e1925cd54b679ddd1406aaf4d9b2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:13.505093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacd4ae-51c2-4d71-913c-cf226b174722 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:13.506555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ebacd4ae-51c2-4d71-913c-cf226b174722 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:13.525692 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebacd4ae-51c2-4d71-913c-cf226b174722 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:13.588005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:13.588634 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:13.588906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:13.589171 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5 admin admin] PATH_INFO: `/v3/projects/27b2d0c6cccf4117ab4e7e7ea3ce8f07` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:13.590286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27b2d0c6cccf4117ab4e7e7ea3ce8f07)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:13.592514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:13.637067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27b2d0c6cccf4117ab4e7e7ea3ce8f07', 'request_id': 'req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:13.642856 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27b2d0c6cccf4117ab4e7e7ea3ce8f07', 'request_id': 'req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:13.650456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27b2d0c6cccf4117ab4e7e7ea3ce8f07', 'request_id': 'req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:13.662521 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5 admin admin] The token cache is being invalidate because project 27b2d0c6cccf4117ab4e7e7ea3ce8f07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:35:13.662809 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:13.663436 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:13.664905 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1272/2560] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:35:13 2022] DELETE /identity/v3/projects/27b2d0c6cccf4117ab4e7e7ea3ce8f07 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:23.330512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=10,keystone:SELECT=22,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:23.658242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:30.731726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4031825-bef7-430c-9e4e-89332906eb8f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:30.733837 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a4031825-bef7-430c-9e4e-89332906eb8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:30.802480 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4031825-bef7-430c-9e4e-89332906eb8f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:30.849500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298253d-1d61-470f-b825-db4301f4adbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:30.850098 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298253d-1d61-470f-b825-db4301f4adbc service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:30.850399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298253d-1d61-470f-b825-db4301f4adbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:30.850671 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3298253d-1d61-470f-b825-db4301f4adbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:30.852442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-3298253d-1d61-470f-b825-db4301f4adbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:30.915054 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298253d-1d61-470f-b825-db4301f4adbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:30.916497 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3298253d-1d61-470f-b825-db4301f4adbc service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:30.926666 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1289/2561] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:34.282495 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:34.284100 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:34.297932 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38778e0-3186-4c7f-b8d9-adc5b2dde8f5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:34.302971 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf8827e-5a4f-4af4-b6c9-f0c3bbd7cc9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:34.303600 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8827e-5a4f-4af4-b6c9-f0c3bbd7cc9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:34.303873 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8827e-5a4f-4af4-b6c9-f0c3bbd7cc9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:34.304138 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf8827e-5a4f-4af4-b6c9-f0c3bbd7cc9e admin admin] PATH_INFO: `/v3/users/461b9a3987744067a62234aaa8d086d0` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:34.305266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8827e-5a4f-4af4-b6c9-f0c3bbd7cc9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=461b9a3987744067a62234aaa8d086d0)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:34.307200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf8827e-5a4f-4af4-b6c9-f0c3bbd7cc9e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:34.353905 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bbf8827e-5a4f-4af4-b6c9-f0c3bbd7cc9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '461b9a3987744067a62234aaa8d086d0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:34.354589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bbf8827e-5a4f-4af4-b6c9-f0c3bbd7cc9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '461b9a3987744067a62234aaa8d086d0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:34.366993 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bbf8827e-5a4f-4af4-b6c9-f0c3bbd7cc9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '461b9a3987744067a62234aaa8d086d0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:34.375415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bbf8827e-5a4f-4af4-b6c9-f0c3bbd7cc9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '461b9a3987744067a62234aaa8d086d0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:34.375975 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-bbf8827e-5a4f-4af4-b6c9-f0c3bbd7cc9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '461b9a3987744067a62234aaa8d086d0'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:34.394212 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1273/2562] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:35:34 2022] DELETE /identity/v3/users/461b9a3987744067a62234aaa8d086d0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:34.575785 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298253d-1d61-470f-b825-db4301f4adbc service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:34.586898 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3298253d-1d61-470f-b825-db4301f4adbc service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:34.646124 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=100 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:34.653273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72e1c1a-5222-43e6-b71d-88291444e85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:34.654461 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e1c1a-5222-43e6-b71d-88291444e85b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:34.654738 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e1c1a-5222-43e6-b71d-88291444e85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:34.655009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72e1c1a-5222-43e6-b71d-88291444e85b admin admin] PATH_INFO: `/v3/projects/ce23bc924f714d72ad20d32a3f5b6d25` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:34.656118 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e1c1a-5222-43e6-b71d-88291444e85b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce23bc924f714d72ad20d32a3f5b6d25)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:34.658030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72e1c1a-5222-43e6-b71d-88291444e85b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:34.704212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e72e1c1a-5222-43e6-b71d-88291444e85b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce23bc924f714d72ad20d32a3f5b6d25', 'request_id': 'req-e72e1c1a-5222-43e6-b71d-88291444e85b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:34.710502 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e72e1c1a-5222-43e6-b71d-88291444e85b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce23bc924f714d72ad20d32a3f5b6d25', 'request_id': 'req-e72e1c1a-5222-43e6-b71d-88291444e85b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:34.718608 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e72e1c1a-5222-43e6-b71d-88291444e85b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce23bc924f714d72ad20d32a3f5b6d25', 'request_id': 'req-e72e1c1a-5222-43e6-b71d-88291444e85b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:34.729175 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e72e1c1a-5222-43e6-b71d-88291444e85b admin admin] The token cache is being invalidate because project ce23bc924f714d72ad20d32a3f5b6d25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:35:34.729443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e72e1c1a-5222-43e6-b71d-88291444e85b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:34.730072 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-e72e1c1a-5222-43e6-b71d-88291444e85b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:34.731600 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1290/2563] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:35:34 2022] DELETE /identity/v3/projects/ce23bc924f714d72ad20d32a3f5b6d25 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:34.737051 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331843b-a7cf-4439-a80a-253a8fdaf5be None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:34.737334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331843b-a7cf-4439-a80a-253a8fdaf5be None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:34.737607 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331843b-a7cf-4439-a80a-253a8fdaf5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:34.795214 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-3331843b-a7cf-4439-a80a-253a8fdaf5be None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:34.796344 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-3331843b-a7cf-4439-a80a-253a8fdaf5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:34.854311 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1274/2564] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:34.862093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7163c2-5ece-4fc3-b7a1-785b8049d13f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:34.862428 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7163c2-5ece-4fc3-b7a1-785b8049d13f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:34.862698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7163c2-5ece-4fc3-b7a1-785b8049d13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:34.882165 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ba7163c2-5ece-4fc3-b7a1-785b8049d13f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:34.884802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ba7163c2-5ece-4fc3-b7a1-785b8049d13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:34.893589 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1291/2565] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:34.897829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331843b-a7cf-4439-a80a-253a8fdaf5be None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:34.907878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331843b-a7cf-4439-a80a-253a8fdaf5be None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:34.912813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690c7dd-ef47-40b9-95fb-caca93e0f4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:34.913401 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690c7dd-ef47-40b9-95fb-caca93e0f4c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:34.913674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690c7dd-ef47-40b9-95fb-caca93e0f4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:34.913942 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690c7dd-ef47-40b9-95fb-caca93e0f4c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:34.914636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690c7dd-ef47-40b9-95fb-caca93e0f4c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:35:34.914933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690c7dd-ef47-40b9-95fb-caca93e0f4c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:34.916561 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a690c7dd-ef47-40b9-95fb-caca93e0f4c1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:34.927813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1275/2566] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:35:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:35:34.932102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7163c2-5ece-4fc3-b7a1-785b8049d13f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:34.941325 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7163c2-5ece-4fc3-b7a1-785b8049d13f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:34.945965 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d70fd5-5f21-43a7-bb07-1c43e4b307af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:34.946610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d70fd5-5f21-43a7-bb07-1c43e4b307af admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:34.946886 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d70fd5-5f21-43a7-bb07-1c43e4b307af admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:34.947156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d70fd5-5f21-43a7-bb07-1c43e4b307af admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:34.947778 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d70fd5-5f21-43a7-bb07-1c43e4b307af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:34.949672 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d70fd5-5f21-43a7-bb07-1c43e4b307af admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:34.977900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1292/2567] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:35:34 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:35:34.982928 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690c7dd-ef47-40b9-95fb-caca93e0f4c1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:34.992666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690c7dd-ef47-40b9-95fb-caca93e0f4c1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:35.019175 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:35.050651 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb21fa6-d9ca-4cec-9b0a-5b9c69594e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:35.051179 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb21fa6-d9ca-4cec-9b0a-5b9c69594e33 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:35.051457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb21fa6-d9ca-4cec-9b0a-5b9c69594e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:35.051727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb21fa6-d9ca-4cec-9b0a-5b9c69594e33 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:35.052337 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb21fa6-d9ca-4cec-9b0a-5b9c69594e33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:35.054249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb21fa6-d9ca-4cec-9b0a-5b9c69594e33 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:35.074860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1276/2568] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:35:34 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:35:35.079032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d70fd5-5f21-43a7-bb07-1c43e4b307af admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:35.089315 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d70fd5-5f21-43a7-bb07-1c43e4b307af admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:35.093957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1043c-fefb-4056-a34b-2426820a7f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:35.094583 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1043c-fefb-4056-a34b-2426820a7f5f admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:35.094863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1043c-fefb-4056-a34b-2426820a7f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:35.095135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf1043c-fefb-4056-a34b-2426820a7f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:35.095780 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1043c-fefb-4056-a34b-2426820a7f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:35.097369 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf1043c-fefb-4056-a34b-2426820a7f5f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:35.107682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1293/2569] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:35.112483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb21fa6-d9ca-4cec-9b0a-5b9c69594e33 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:35.124157 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb21fa6-d9ca-4cec-9b0a-5b9c69594e33 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:35.128779 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3cf472-2262-4491-94a4-27665d86fd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:35.129377 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cf472-2262-4491-94a4-27665d86fd40 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:35.129657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cf472-2262-4491-94a4-27665d86fd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:35.130040 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cf472-2262-4491-94a4-27665d86fd40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:35.130725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3cf472-2262-4491-94a4-27665d86fd40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:35.132320 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3cf472-2262-4491-94a4-27665d86fd40 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:35.142878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1277/2570] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:35.147206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1043c-fefb-4056-a34b-2426820a7f5f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:35.157330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf1043c-fefb-4056-a34b-2426820a7f5f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:35.162427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b80b59-4e1b-407b-ab36-277f5c6801ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:35.163070 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b80b59-4e1b-407b-ab36-277f5c6801ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:35.163347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b80b59-4e1b-407b-ab36-277f5c6801ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:35.163618 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b80b59-4e1b-407b-ab36-277f5c6801ae admin admin] PATH_INFO: `/v3/projects/e24be09b13ce4a509424f46c08cc022b/users/c1f6a1e7dc664ce29e65e06d96559784/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:35.178572 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b80b59-4e1b-407b-ab36-277f5c6801ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=e24be09b13ce4a509424f46c08cc022b, user_id=c1f6a1e7dc664ce29e65e06d96559784, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:35.181213 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b80b59-4e1b-407b-ab36-277f5c6801ae admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:35.194606 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1294/2571] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:35:35 2022] PUT /identity/v3/projects/e24be09b13ce4a509424f46c08cc022b/users/c1f6a1e7dc664ce29e65e06d96559784/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:35.204908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acece34b-d1ce-447c-b452-c668dfc5d28e None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:35.205198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acece34b-d1ce-447c-b452-c668dfc5d28e None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:35.205473 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acece34b-d1ce-447c-b452-c668dfc5d28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:35.233630 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-acece34b-d1ce-447c-b452-c668dfc5d28e None None] MFA Rules not processed for user `c1f6a1e7dc664ce29e65e06d96559784`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:35.234886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-acece34b-d1ce-447c-b452-c668dfc5d28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:35.297696 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1278/2572] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:35 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:35.302612 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b80b59-4e1b-407b-ab36-277f5c6801ae admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:35.368686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b80b59-4e1b-407b-ab36-277f5c6801ae admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:35.373454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849c729-c2f6-4875-9589-a0590204ace1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:35.374064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849c729-c2f6-4875-9589-a0590204ace1 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:35.374386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849c729-c2f6-4875-9589-a0590204ace1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:35.374679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3849c729-c2f6-4875-9589-a0590204ace1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:35.375288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3849c729-c2f6-4875-9589-a0590204ace1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:35.377226 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3849c729-c2f6-4875-9589-a0590204ace1 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:35.403069 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1295/2573] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:35:35 2022] POST /identity/v3/projects => generated 405 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:35:35.407704 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acece34b-d1ce-447c-b452-c668dfc5d28e None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:35.418753 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acece34b-d1ce-447c-b452-c668dfc5d28e None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:35.474098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e6a45-8429-4d83-9801-7dca8276359b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:35.474734 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e6a45-8429-4d83-9801-7dca8276359b admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:35.475016 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e6a45-8429-4d83-9801-7dca8276359b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:35.475292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9e6a45-8429-4d83-9801-7dca8276359b admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:35.475914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e6a45-8429-4d83-9801-7dca8276359b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:35.477827 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9e6a45-8429-4d83-9801-7dca8276359b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:35.500606 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1279/2574] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:35:35 2022] POST /identity/v3/users => generated 309 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:35:35.504752 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849c729-c2f6-4875-9589-a0590204ace1 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:35.515654 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3849c729-c2f6-4875-9589-a0590204ace1 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:35.520313 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72021831-6470-447a-a8a6-4c5cc758c9d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:35.520900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72021831-6470-447a-a8a6-4c5cc758c9d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:35.521170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72021831-6470-447a-a8a6-4c5cc758c9d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:35.521443 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72021831-6470-447a-a8a6-4c5cc758c9d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:35.522082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72021831-6470-447a-a8a6-4c5cc758c9d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:35.523693 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72021831-6470-447a-a8a6-4c5cc758c9d8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:35.524933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:35.541350 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1296/2575] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:35:35.542364 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e6a45-8429-4d83-9801-7dca8276359b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:35.553842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9e6a45-8429-4d83-9801-7dca8276359b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:35.558443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a7243-7b73-4321-95bb-04469e59680d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:35.559095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a7243-7b73-4321-95bb-04469e59680d admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:35.559382 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a7243-7b73-4321-95bb-04469e59680d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:35.559657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412a7243-7b73-4321-95bb-04469e59680d admin admin] PATH_INFO: `/v3/projects/89af29c9d2704fd89cefeb5d52d9a64d/users/75ee03eb129642b281e3270c9534a935/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:35.575336 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a7243-7b73-4321-95bb-04469e59680d admin admin] RBAC: Authorizing `identity:create_grant(project_id=89af29c9d2704fd89cefeb5d52d9a64d, user_id=75ee03eb129642b281e3270c9534a935, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:35.578596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412a7243-7b73-4321-95bb-04469e59680d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:35.591791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1280/2576] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:35:35 2022] PUT /identity/v3/projects/89af29c9d2704fd89cefeb5d52d9a64d/users/75ee03eb129642b281e3270c9534a935/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:35:35.595888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72021831-6470-447a-a8a6-4c5cc758c9d8 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:35.606038 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72021831-6470-447a-a8a6-4c5cc758c9d8 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:35.663851 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b140bd2f-77d9-4821-a460-00f5b6192e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:35.664506 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140bd2f-77d9-4821-a460-00f5b6192e99 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:35.664781 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140bd2f-77d9-4821-a460-00f5b6192e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:35.665049 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b140bd2f-77d9-4821-a460-00f5b6192e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:35.665707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140bd2f-77d9-4821-a460-00f5b6192e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:35.667326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b140bd2f-77d9-4821-a460-00f5b6192e99 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:35.677794 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1297/2577] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:35 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:35.682007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a7243-7b73-4321-95bb-04469e59680d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:35.688046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:35.694644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412a7243-7b73-4321-95bb-04469e59680d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:35.699665 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2eb7c3-24c9-4480-b5cf-3ead8d129713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:35.700336 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2eb7c3-24c9-4480-b5cf-3ead8d129713 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:35.700632 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2eb7c3-24c9-4480-b5cf-3ead8d129713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:35.700897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2eb7c3-24c9-4480-b5cf-3ead8d129713 admin admin] PATH_INFO: `/v3/projects/89af29c9d2704fd89cefeb5d52d9a64d/users/75ee03eb129642b281e3270c9534a935/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:35.702793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2eb7c3-24c9-4480-b5cf-3ead8d129713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89af29c9d2704fd89cefeb5d52d9a64d, user_id=75ee03eb129642b281e3270c9534a935, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:35.705370 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2eb7c3-24c9-4480-b5cf-3ead8d129713 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:35.715273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1281/2578] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:35:35 2022] PUT /identity/v3/projects/89af29c9d2704fd89cefeb5d52d9a64d/users/75ee03eb129642b281e3270c9534a935/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:35.726732 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23c1a0-2080-429f-888e-c500dbefe534 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:35.727014 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23c1a0-2080-429f-888e-c500dbefe534 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:35.727283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23c1a0-2080-429f-888e-c500dbefe534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:35.755373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-bf23c1a0-2080-429f-888e-c500dbefe534 None None] MFA Rules not processed for user `75ee03eb129642b281e3270c9534a935`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:35.756590 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bf23c1a0-2080-429f-888e-c500dbefe534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:35.824240 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1298/2579] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:35 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:35.834881 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2eb7c3-24c9-4480-b5cf-3ead8d129713 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:35.836572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9c2eb7c3-24c9-4480-b5cf-3ead8d129713 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:35.915667 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2eb7c3-24c9-4480-b5cf-3ead8d129713 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:35.919981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f26f3-bc44-418c-8d47-72bb251ce78a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:35.920583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f26f3-bc44-418c-8d47-72bb251ce78a service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:35.920862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f26f3-bc44-418c-8d47-72bb251ce78a service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:35.921124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f26f3-bc44-418c-8d47-72bb251ce78a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:35.930181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f26f3-bc44-418c-8d47-72bb251ce78a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:35.931611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f26f3-bc44-418c-8d47-72bb251ce78a service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:35.941450 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1282/2580] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:35 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:36.077488 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23c1a0-2080-429f-888e-c500dbefe534 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:36.140093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf23c1a0-2080-429f-888e-c500dbefe534 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:36.144944 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5cd59-ddfc-4bc5-b2e2-2a1b17f93427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:36.145605 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5cd59-ddfc-4bc5-b2e2-2a1b17f93427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:36.145883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5cd59-ddfc-4bc5-b2e2-2a1b17f93427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:36.146153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d5cd59-ddfc-4bc5-b2e2-2a1b17f93427 admin admin] PATH_INFO: `/v3/users/c1f6a1e7dc664ce29e65e06d96559784` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:36.147322 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5cd59-ddfc-4bc5-b2e2-2a1b17f93427 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1f6a1e7dc664ce29e65e06d96559784)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:36.149231 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d5cd59-ddfc-4bc5-b2e2-2a1b17f93427 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:36.171220 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:36.195661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-60d5cd59-ddfc-4bc5-b2e2-2a1b17f93427 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1f6a1e7dc664ce29e65e06d96559784'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:36.203271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-60d5cd59-ddfc-4bc5-b2e2-2a1b17f93427 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1f6a1e7dc664ce29e65e06d96559784'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:36.215187 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-60d5cd59-ddfc-4bc5-b2e2-2a1b17f93427 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1f6a1e7dc664ce29e65e06d96559784'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:36.215847 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-60d5cd59-ddfc-4bc5-b2e2-2a1b17f93427 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1f6a1e7dc664ce29e65e06d96559784'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:36.223253 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-60d5cd59-ddfc-4bc5-b2e2-2a1b17f93427 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1f6a1e7dc664ce29e65e06d96559784'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:36.229087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1299/2581] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:35:36 2022] DELETE /identity/v3/users/c1f6a1e7dc664ce29e65e06d96559784 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:36.234183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ca043-3e74-4608-9a5c-d8b320afaa85 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:36.234510 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ca043-3e74-4608-9a5c-d8b320afaa85 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:36.234780 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32ca043-3e74-4608-9a5c-d8b320afaa85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:36.253189 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b32ca043-3e74-4608-9a5c-d8b320afaa85 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:36.254477 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b32ca043-3e74-4608-9a5c-d8b320afaa85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:36.313753 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1283/2582] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:36.323350 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5cd59-ddfc-4bc5-b2e2-2a1b17f93427 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:36.324729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-60d5cd59-ddfc-4bc5-b2e2-2a1b17f93427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:36.381478 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d5cd59-ddfc-4bc5-b2e2-2a1b17f93427 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:36.385048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01a41e-0ab0-429b-b351-558584c27509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:36.385640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01a41e-0ab0-429b-b351-558584c27509 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:36.385915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01a41e-0ab0-429b-b351-558584c27509 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:36.386180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01a41e-0ab0-429b-b351-558584c27509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:36.395818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01a41e-0ab0-429b-b351-558584c27509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:36.397246 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f01a41e-0ab0-429b-b351-558584c27509 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:36.407418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1300/2583] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:36.676405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ca043-3e74-4608-9a5c-d8b320afaa85 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:36.677974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b32ca043-3e74-4608-9a5c-d8b320afaa85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:36.691428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32ca043-3e74-4608-9a5c-d8b320afaa85 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:36.696054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3296630-ae0c-4e0d-9382-e5c1d862b587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:36.696689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3296630-ae0c-4e0d-9382-e5c1d862b587 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:36.696973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3296630-ae0c-4e0d-9382-e5c1d862b587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:36.697246 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3296630-ae0c-4e0d-9382-e5c1d862b587 admin admin] PATH_INFO: `/v3/projects/e24be09b13ce4a509424f46c08cc022b` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:36.698395 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3296630-ae0c-4e0d-9382-e5c1d862b587 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e24be09b13ce4a509424f46c08cc022b)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:36.700413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3296630-ae0c-4e0d-9382-e5c1d862b587 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:36.747075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d3296630-ae0c-4e0d-9382-e5c1d862b587 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e24be09b13ce4a509424f46c08cc022b', 'request_id': 'req-d3296630-ae0c-4e0d-9382-e5c1d862b587'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:36.752435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d3296630-ae0c-4e0d-9382-e5c1d862b587 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e24be09b13ce4a509424f46c08cc022b', 'request_id': 'req-d3296630-ae0c-4e0d-9382-e5c1d862b587'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:36.759684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d3296630-ae0c-4e0d-9382-e5c1d862b587 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e24be09b13ce4a509424f46c08cc022b', 'request_id': 'req-d3296630-ae0c-4e0d-9382-e5c1d862b587'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:36.761579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:36.772012 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d3296630-ae0c-4e0d-9382-e5c1d862b587 admin admin] The token cache is being invalidate because project e24be09b13ce4a509424f46c08cc022b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:35:36.772321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d3296630-ae0c-4e0d-9382-e5c1d862b587 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:36.772955 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-d3296630-ae0c-4e0d-9382-e5c1d862b587 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:36.774472 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1284/2584] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:35:36 2022] DELETE /identity/v3/projects/e24be09b13ce4a509424f46c08cc022b => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:36.778979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01a41e-0ab0-429b-b351-558584c27509 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:36.788846 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f01a41e-0ab0-429b-b351-558584c27509 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:36.844678 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0687010-55dc-463d-b231-33f2f3c2423f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:36.845303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0687010-55dc-463d-b231-33f2f3c2423f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:36.845583 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0687010-55dc-463d-b231-33f2f3c2423f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:36.845858 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0687010-55dc-463d-b231-33f2f3c2423f admin admin] PATH_INFO: `/v3/users/75ee03eb129642b281e3270c9534a935` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:36.846996 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0687010-55dc-463d-b231-33f2f3c2423f admin admin] RBAC: Authorizing `identity:delete_user(user_id=75ee03eb129642b281e3270c9534a935)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:36.848927 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0687010-55dc-463d-b231-33f2f3c2423f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:36.889005 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:36.894459 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d0687010-55dc-463d-b231-33f2f3c2423f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75ee03eb129642b281e3270c9534a935'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:36.902425 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d0687010-55dc-463d-b231-33f2f3c2423f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75ee03eb129642b281e3270c9534a935'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:36.921280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d0687010-55dc-463d-b231-33f2f3c2423f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75ee03eb129642b281e3270c9534a935'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:36.921871 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d0687010-55dc-463d-b231-33f2f3c2423f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75ee03eb129642b281e3270c9534a935'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:36.928927 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-d0687010-55dc-463d-b231-33f2f3c2423f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75ee03eb129642b281e3270c9534a935'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:36.934799 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1301/2585] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:35:36 2022] DELETE /identity/v3/users/75ee03eb129642b281e3270c9534a935 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:37.213662 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3296630-ae0c-4e0d-9382-e5c1d862b587 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:37.215306 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d3296630-ae0c-4e0d-9382-e5c1d862b587 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:37.231906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3296630-ae0c-4e0d-9382-e5c1d862b587 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:37.291089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78191f86-a5e9-4bdb-b2e4-9047a70ea93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:37.291737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78191f86-a5e9-4bdb-b2e4-9047a70ea93f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:37.292010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78191f86-a5e9-4bdb-b2e4-9047a70ea93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:37.292276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78191f86-a5e9-4bdb-b2e4-9047a70ea93f admin admin] PATH_INFO: `/v3/projects/89af29c9d2704fd89cefeb5d52d9a64d` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:37.293376 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78191f86-a5e9-4bdb-b2e4-9047a70ea93f admin admin] RBAC: Authorizing `identity:delete_project(project_id=89af29c9d2704fd89cefeb5d52d9a64d)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:37.295308 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78191f86-a5e9-4bdb-b2e4-9047a70ea93f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:37.340957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-78191f86-a5e9-4bdb-b2e4-9047a70ea93f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89af29c9d2704fd89cefeb5d52d9a64d', 'request_id': 'req-78191f86-a5e9-4bdb-b2e4-9047a70ea93f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:37.347078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-78191f86-a5e9-4bdb-b2e4-9047a70ea93f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89af29c9d2704fd89cefeb5d52d9a64d', 'request_id': 'req-78191f86-a5e9-4bdb-b2e4-9047a70ea93f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:37.354235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-78191f86-a5e9-4bdb-b2e4-9047a70ea93f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89af29c9d2704fd89cefeb5d52d9a64d', 'request_id': 'req-78191f86-a5e9-4bdb-b2e4-9047a70ea93f'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:37.365488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-78191f86-a5e9-4bdb-b2e4-9047a70ea93f admin admin] The token cache is being invalidate because project 89af29c9d2704fd89cefeb5d52d9a64d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:35:37.365777 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-78191f86-a5e9-4bdb-b2e4-9047a70ea93f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:37.366442 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-78191f86-a5e9-4bdb-b2e4-9047a70ea93f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:37.367911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1285/2586] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:35:37 2022] DELETE /identity/v3/projects/89af29c9d2704fd89cefeb5d52d9a64d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:37.372777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a35ae-623e-41e0-9ac3-116f3216f6f9 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:37.373058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a35ae-623e-41e0-9ac3-116f3216f6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:37.373326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a35ae-623e-41e0-9ac3-116f3216f6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:37.394790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e00a35ae-623e-41e0-9ac3-116f3216f6f9 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:37.396000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e00a35ae-623e-41e0-9ac3-116f3216f6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:37.454638 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1302/2587] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:37.462932 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88c3f8-1997-4fa8-b3a9-58e675b39410 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:37.463154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88c3f8-1997-4fa8-b3a9-58e675b39410 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:37.463429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88c3f8-1997-4fa8-b3a9-58e675b39410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:37.482910 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-ef88c3f8-1997-4fa8-b3a9-58e675b39410 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:37.484122 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ef88c3f8-1997-4fa8-b3a9-58e675b39410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:37.492001 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1286/2588] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:37.496419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a35ae-623e-41e0-9ac3-116f3216f6f9 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:37.507166 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a35ae-623e-41e0-9ac3-116f3216f6f9 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:37.511727 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93beecc8-59d1-4a70-bfb7-0555ad55a553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:37.512322 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beecc8-59d1-4a70-bfb7-0555ad55a553 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:37.512597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beecc8-59d1-4a70-bfb7-0555ad55a553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:37.512915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93beecc8-59d1-4a70-bfb7-0555ad55a553 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:37.514100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93beecc8-59d1-4a70-bfb7-0555ad55a553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:35:37.514430 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93beecc8-59d1-4a70-bfb7-0555ad55a553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:37.516018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93beecc8-59d1-4a70-bfb7-0555ad55a553 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:37.527971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1303/2589] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:35:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:35:37.532534 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88c3f8-1997-4fa8-b3a9-58e675b39410 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:37.542328 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88c3f8-1997-4fa8-b3a9-58e675b39410 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:37.547155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366a431-add1-4cf5-9fc8-c970379e047f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:37.547799 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366a431-add1-4cf5-9fc8-c970379e047f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:37.548076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366a431-add1-4cf5-9fc8-c970379e047f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:37.548341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3366a431-add1-4cf5-9fc8-c970379e047f admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:37.548958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366a431-add1-4cf5-9fc8-c970379e047f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:37.550942 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3366a431-add1-4cf5-9fc8-c970379e047f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:37.578238 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1287/2590] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:35:37 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:35:37.582337 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93beecc8-59d1-4a70-bfb7-0555ad55a553 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:37.592538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93beecc8-59d1-4a70-bfb7-0555ad55a553 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:37.650078 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814aeb28-7a95-483f-be6c-5f71dcbbd85b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:37.650724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814aeb28-7a95-483f-be6c-5f71dcbbd85b admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:37.651004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814aeb28-7a95-483f-be6c-5f71dcbbd85b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:37.651281 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814aeb28-7a95-483f-be6c-5f71dcbbd85b admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:37.651903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814aeb28-7a95-483f-be6c-5f71dcbbd85b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:37.653874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814aeb28-7a95-483f-be6c-5f71dcbbd85b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:37.675538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1304/2591] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:35:37 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:35:37.679655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366a431-add1-4cf5-9fc8-c970379e047f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:37.690480 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3366a431-add1-4cf5-9fc8-c970379e047f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:37.695347 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d076a5c-6aa4-4b80-b6db-2db6f21cfa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:37.695940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d076a5c-6aa4-4b80-b6db-2db6f21cfa2a admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:37.696218 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d076a5c-6aa4-4b80-b6db-2db6f21cfa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:37.696489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d076a5c-6aa4-4b80-b6db-2db6f21cfa2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:37.697127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d076a5c-6aa4-4b80-b6db-2db6f21cfa2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:37.698740 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d076a5c-6aa4-4b80-b6db-2db6f21cfa2a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:37.709209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1288/2592] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:37.713461 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814aeb28-7a95-483f-be6c-5f71dcbbd85b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:37.723770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814aeb28-7a95-483f-be6c-5f71dcbbd85b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:37.728500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145a068-0b62-40f9-893e-f323ede82113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:37.729091 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145a068-0b62-40f9-893e-f323ede82113 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:37.729373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145a068-0b62-40f9-893e-f323ede82113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:37.729645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145a068-0b62-40f9-893e-f323ede82113 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:37.730316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145a068-0b62-40f9-893e-f323ede82113 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:37.731932 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145a068-0b62-40f9-893e-f323ede82113 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:37.742471 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1305/2593] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:37.746474 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d076a5c-6aa4-4b80-b6db-2db6f21cfa2a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:37.757061 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d076a5c-6aa4-4b80-b6db-2db6f21cfa2a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:37.761727 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686eec7b-1b3f-46f3-82a9-403d21dc686d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:37.762464 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eec7b-1b3f-46f3-82a9-403d21dc686d admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:37.762746 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eec7b-1b3f-46f3-82a9-403d21dc686d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:37.763621 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686eec7b-1b3f-46f3-82a9-403d21dc686d admin admin] PATH_INFO: `/v3/projects/786c60747bf748eab9a907c2ce1d38b6/users/5ba6a4ce080049e1a89c22d77911d4cf/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:37.777940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686eec7b-1b3f-46f3-82a9-403d21dc686d admin admin] RBAC: Authorizing `identity:create_grant(project_id=786c60747bf748eab9a907c2ce1d38b6, user_id=5ba6a4ce080049e1a89c22d77911d4cf, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:37.780617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686eec7b-1b3f-46f3-82a9-403d21dc686d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:37.794136 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1289/2594] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:35:37 2022] PUT /identity/v3/projects/786c60747bf748eab9a907c2ce1d38b6/users/5ba6a4ce080049e1a89c22d77911d4cf/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:37.804439 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de121944-5894-4959-b35a-70d1866d1df0 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:37.804925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de121944-5894-4959-b35a-70d1866d1df0 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:37.805024 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de121944-5894-4959-b35a-70d1866d1df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:37.833175 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-de121944-5894-4959-b35a-70d1866d1df0 None None] MFA Rules not processed for user `5ba6a4ce080049e1a89c22d77911d4cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:37.834433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-de121944-5894-4959-b35a-70d1866d1df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:37.846042 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:37.900118 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1306/2595] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:37 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:37.904612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686eec7b-1b3f-46f3-82a9-403d21dc686d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:37.929988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:37.967493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686eec7b-1b3f-46f3-82a9-403d21dc686d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:37.972060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c1ee9-a770-4d17-bdc2-91aaae91b20b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:37.972656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c1ee9-a770-4d17-bdc2-91aaae91b20b admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:37.972930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c1ee9-a770-4d17-bdc2-91aaae91b20b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:37.973197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c1ee9-a770-4d17-bdc2-91aaae91b20b admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:37.973815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c1ee9-a770-4d17-bdc2-91aaae91b20b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:37.975721 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c1ee9-a770-4d17-bdc2-91aaae91b20b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:38.002403 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1290/2596] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:35:37 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:35:38.006581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de121944-5894-4959-b35a-70d1866d1df0 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:38.017715 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de121944-5894-4959-b35a-70d1866d1df0 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:38.079133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc84d1-b9b4-4e62-a7e9-5edca13d6647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:38.079736 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc84d1-b9b4-4e62-a7e9-5edca13d6647 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:38.080007 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc84d1-b9b4-4e62-a7e9-5edca13d6647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:38.080273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc84d1-b9b4-4e62-a7e9-5edca13d6647 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:38.080878 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc84d1-b9b4-4e62-a7e9-5edca13d6647 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:38.082832 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37bc84d1-b9b4-4e62-a7e9-5edca13d6647 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:38.104510 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1307/2597] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:35:38 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:35:38.109028 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c1ee9-a770-4d17-bdc2-91aaae91b20b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:38.119862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c1ee9-a770-4d17-bdc2-91aaae91b20b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:38.125149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e828740-9ea6-4d47-be47-30ec2d69d09f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:38.125785 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e828740-9ea6-4d47-be47-30ec2d69d09f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:38.126062 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e828740-9ea6-4d47-be47-30ec2d69d09f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:38.126381 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e828740-9ea6-4d47-be47-30ec2d69d09f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:38.127025 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e828740-9ea6-4d47-be47-30ec2d69d09f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:38.128624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e828740-9ea6-4d47-be47-30ec2d69d09f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:38.139951 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1291/2598] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:38.144382 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc84d1-b9b4-4e62-a7e9-5edca13d6647 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:38.154475 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc84d1-b9b4-4e62-a7e9-5edca13d6647 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:38.159151 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c6de1-f983-4cfb-bd3b-90c01ee7a37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:38.160005 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c6de1-f983-4cfb-bd3b-90c01ee7a37d admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:38.160280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c6de1-f983-4cfb-bd3b-90c01ee7a37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:38.160542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c6de1-f983-4cfb-bd3b-90c01ee7a37d admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:38.161172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c6de1-f983-4cfb-bd3b-90c01ee7a37d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:38.162799 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c6de1-f983-4cfb-bd3b-90c01ee7a37d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:38.173206 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1308/2599] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:38.177399 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e828740-9ea6-4d47-be47-30ec2d69d09f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:38.187780 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e828740-9ea6-4d47-be47-30ec2d69d09f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:38.192388 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c47e2-f284-49f8-b67e-13e8f3dac310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:38.193023 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c47e2-f284-49f8-b67e-13e8f3dac310 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:38.193294 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c47e2-f284-49f8-b67e-13e8f3dac310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:38.193558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c47e2-f284-49f8-b67e-13e8f3dac310 admin admin] PATH_INFO: `/v3/projects/06c8d005608647d996f4b277a5344a0e/users/0f01a7a7d33d48fd97f969ab9e80b1e9/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:38.207688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c47e2-f284-49f8-b67e-13e8f3dac310 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06c8d005608647d996f4b277a5344a0e, user_id=0f01a7a7d33d48fd97f969ab9e80b1e9, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:38.210310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c47e2-f284-49f8-b67e-13e8f3dac310 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:38.224671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1292/2600] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:35:38 2022] PUT /identity/v3/projects/06c8d005608647d996f4b277a5344a0e/users/0f01a7a7d33d48fd97f969ab9e80b1e9/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:38.236037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cfd7b-1c4b-418a-990f-c062bfd2cf03 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:38.236327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cfd7b-1c4b-418a-990f-c062bfd2cf03 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:38.236609 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0cfd7b-1c4b-418a-990f-c062bfd2cf03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:38.264983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7d0cfd7b-1c4b-418a-990f-c062bfd2cf03 None None] MFA Rules not processed for user `0f01a7a7d33d48fd97f969ab9e80b1e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:38.266191 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7d0cfd7b-1c4b-418a-990f-c062bfd2cf03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:38.329640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1309/2601] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:38 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:38.334309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c47e2-f284-49f8-b67e-13e8f3dac310 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:38.394850 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c47e2-f284-49f8-b67e-13e8f3dac310 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:38.399677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfe3f5-232d-4918-9ac2-28c0bdc2932c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:38.400280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfe3f5-232d-4918-9ac2-28c0bdc2932c admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:38.400568 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfe3f5-232d-4918-9ac2-28c0bdc2932c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:38.400840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cfe3f5-232d-4918-9ac2-28c0bdc2932c admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:38.401472 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfe3f5-232d-4918-9ac2-28c0bdc2932c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:38.403434 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cfe3f5-232d-4918-9ac2-28c0bdc2932c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:38.429639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1293/2602] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:35:38 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:35:38.433685 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cfd7b-1c4b-418a-990f-c062bfd2cf03 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:38.445084 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0cfd7b-1c4b-418a-990f-c062bfd2cf03 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:38.463271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:38.501369 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34d7de-8c4d-4b0d-9b2a-3331a211d278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:38.502158 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34d7de-8c4d-4b0d-9b2a-3331a211d278 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:38.502158 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34d7de-8c4d-4b0d-9b2a-3331a211d278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:38.502449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b34d7de-8c4d-4b0d-9b2a-3331a211d278 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:38.503070 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34d7de-8c4d-4b0d-9b2a-3331a211d278 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:38.504978 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b34d7de-8c4d-4b0d-9b2a-3331a211d278 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:38.525817 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1310/2603] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:35:38 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:35:38.531148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfe3f5-232d-4918-9ac2-28c0bdc2932c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:38.541745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cfe3f5-232d-4918-9ac2-28c0bdc2932c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:38.546986 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2d5d6-49a9-4533-b4af-449271401430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:38.547579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2d5d6-49a9-4533-b4af-449271401430 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:38.547853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2d5d6-49a9-4533-b4af-449271401430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:38.548120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f2d5d6-49a9-4533-b4af-449271401430 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:38.548816 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2d5d6-49a9-4533-b4af-449271401430 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:38.550715 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f2d5d6-49a9-4533-b4af-449271401430 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:38.561818 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1294/2604] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:38.567006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34d7de-8c4d-4b0d-9b2a-3331a211d278 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:38.577650 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b34d7de-8c4d-4b0d-9b2a-3331a211d278 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:38.582300 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449f804-9976-4ceb-91be-0544cf8c02bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:38.582944 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449f804-9976-4ceb-91be-0544cf8c02bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:38.583222 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449f804-9976-4ceb-91be-0544cf8c02bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:38.583493 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2449f804-9976-4ceb-91be-0544cf8c02bc admin admin] PATH_INFO: `/v3/projects/b44affdacd3b41daaf413a57ad348a93/users/90bf41cb44d946faab66c6830def7cf6/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:38.597822 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449f804-9976-4ceb-91be-0544cf8c02bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44affdacd3b41daaf413a57ad348a93, user_id=90bf41cb44d946faab66c6830def7cf6, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:38.600496 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2449f804-9976-4ceb-91be-0544cf8c02bc admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:38.613968 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1311/2605] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:35:38 2022] PUT /identity/v3/projects/b44affdacd3b41daaf413a57ad348a93/users/90bf41cb44d946faab66c6830def7cf6/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:38.618064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2d5d6-49a9-4533-b4af-449271401430 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:38.628616 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f2d5d6-49a9-4533-b4af-449271401430 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:38.673339 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:38.688087 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505320dd-2908-42f9-8987-5158720525b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:38.688688 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505320dd-2908-42f9-8987-5158720525b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:38.688962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505320dd-2908-42f9-8987-5158720525b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:38.689226 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505320dd-2908-42f9-8987-5158720525b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:38.689848 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505320dd-2908-42f9-8987-5158720525b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:38.691471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505320dd-2908-42f9-8987-5158720525b3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:38.701693 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1295/2606] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:38.705938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449f804-9976-4ceb-91be-0544cf8c02bc admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:38.716794 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2449f804-9976-4ceb-91be-0544cf8c02bc admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:38.721423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5618d-181a-4142-94ca-5e90cb7e0c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:38.722064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5618d-181a-4142-94ca-5e90cb7e0c4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:38.722372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5618d-181a-4142-94ca-5e90cb7e0c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:38.722643 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f5618d-181a-4142-94ca-5e90cb7e0c4f admin admin] PATH_INFO: `/v3/projects/b44affdacd3b41daaf413a57ad348a93/users/90bf41cb44d946faab66c6830def7cf6/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:38.724568 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5618d-181a-4142-94ca-5e90cb7e0c4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b44affdacd3b41daaf413a57ad348a93, user_id=90bf41cb44d946faab66c6830def7cf6, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:38.727385 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f5618d-181a-4142-94ca-5e90cb7e0c4f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:38.737505 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1312/2607] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:35:38 2022] PUT /identity/v3/projects/b44affdacd3b41daaf413a57ad348a93/users/90bf41cb44d946faab66c6830def7cf6/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:38.747838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3ea6e-578f-4de8-a0cd-79d29b1f3268 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:38.748124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3ea6e-578f-4de8-a0cd-79d29b1f3268 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:38.748399 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da3ea6e-578f-4de8-a0cd-79d29b1f3268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:38.775861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7da3ea6e-578f-4de8-a0cd-79d29b1f3268 None None] MFA Rules not processed for user `90bf41cb44d946faab66c6830def7cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:38.776995 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7da3ea6e-578f-4de8-a0cd-79d29b1f3268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:38.850881 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1296/2608] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:38.862346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5618d-181a-4142-94ca-5e90cb7e0c4f admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:38.864144 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-43f5618d-181a-4142-94ca-5e90cb7e0c4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:38.923287 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f5618d-181a-4142-94ca-5e90cb7e0c4f admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:38.927146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f215709-9159-42d4-9886-94d31a263540 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:38.927740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f215709-9159-42d4-9886-94d31a263540 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:38.928012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f215709-9159-42d4-9886-94d31a263540 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:38.928277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f215709-9159-42d4-9886-94d31a263540 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:38.937840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f215709-9159-42d4-9886-94d31a263540 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:38.939303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f215709-9159-42d4-9886-94d31a263540 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:38.948772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1313/2609] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:38 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:39.446445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3ea6e-578f-4de8-a0cd-79d29b1f3268 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:39.447825 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7da3ea6e-578f-4de8-a0cd-79d29b1f3268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:39.459571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da3ea6e-578f-4de8-a0cd-79d29b1f3268 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:39.463104 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a53e9-2f3a-4da5-ad3b-45d691117ceb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:39.463694 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a53e9-2f3a-4da5-ad3b-45d691117ceb service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:39.463971 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a53e9-2f3a-4da5-ad3b-45d691117ceb service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:39.464240 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679a53e9-2f3a-4da5-ad3b-45d691117ceb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:39.521494 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a53e9-2f3a-4da5-ad3b-45d691117ceb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:39.522941 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679a53e9-2f3a-4da5-ad3b-45d691117ceb service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:39.531851 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1297/2610] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:39 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:43.531348 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f215709-9159-42d4-9886-94d31a263540 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:43.540480 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f215709-9159-42d4-9886-94d31a263540 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:43.544291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef3b57-5c1c-4b0b-99f1-36858aee1699 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:43.544878 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef3b57-5c1c-4b0b-99f1-36858aee1699 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:43.545177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef3b57-5c1c-4b0b-99f1-36858aee1699 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:43.545516 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef3b57-5c1c-4b0b-99f1-36858aee1699 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:43.602241 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef3b57-5c1c-4b0b-99f1-36858aee1699 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:43.603594 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ef3b57-5c1c-4b0b-99f1-36858aee1699 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:43.614263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1314/2611] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:43 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:45.674385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a53e9-2f3a-4da5-ad3b-45d691117ceb service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:45.740310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679a53e9-2f3a-4da5-ad3b-45d691117ceb service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:45.745037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f2a99-36c6-435b-960f-94bd37744e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:45.745657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f2a99-36c6-435b-960f-94bd37744e93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:45.745934 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f2a99-36c6-435b-960f-94bd37744e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:45.746202 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0f2a99-36c6-435b-960f-94bd37744e93 admin admin] PATH_INFO: `/v3/users/5ba6a4ce080049e1a89c22d77911d4cf` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:45.747339 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f2a99-36c6-435b-960f-94bd37744e93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ba6a4ce080049e1a89c22d77911d4cf)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:45.749245 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0f2a99-36c6-435b-960f-94bd37744e93 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:45.754519 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:45.793694 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0f0f2a99-36c6-435b-960f-94bd37744e93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ba6a4ce080049e1a89c22d77911d4cf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:45.794900 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0f0f2a99-36c6-435b-960f-94bd37744e93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ba6a4ce080049e1a89c22d77911d4cf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:45.806748 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0f0f2a99-36c6-435b-960f-94bd37744e93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ba6a4ce080049e1a89c22d77911d4cf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:45.814271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0f0f2a99-36c6-435b-960f-94bd37744e93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ba6a4ce080049e1a89c22d77911d4cf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:45.814901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-0f0f2a99-36c6-435b-960f-94bd37744e93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ba6a4ce080049e1a89c22d77911d4cf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:45.827065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1298/2612] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:35:45 2022] DELETE /identity/v3/users/5ba6a4ce080049e1a89c22d77911d4cf => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:45.831705 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3167e-6c2e-4252-977f-e6c3334a217f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:45.832197 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3167e-6c2e-4252-977f-e6c3334a217f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:45.832315 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3167e-6c2e-4252-977f-e6c3334a217f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:45.851708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d2d3167e-6c2e-4252-977f-e6c3334a217f None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:45.852901 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d2d3167e-6c2e-4252-977f-e6c3334a217f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:45.862559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:45.915129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1315/2613] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:45.928160 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f2a99-36c6-435b-960f-94bd37744e93 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:45.929572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0f0f2a99-36c6-435b-960f-94bd37744e93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:45.989408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0f2a99-36c6-435b-960f-94bd37744e93 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:45.993095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984581b-6c7b-4e3d-9a67-3362a93011e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:45.993729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984581b-6c7b-4e3d-9a67-3362a93011e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:45.993979 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984581b-6c7b-4e3d-9a67-3362a93011e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:45.994277 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d984581b-6c7b-4e3d-9a67-3362a93011e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:46.004513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984581b-6c7b-4e3d-9a67-3362a93011e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:46.005935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d984581b-6c7b-4e3d-9a67-3362a93011e7 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:46.015497 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1299/2614] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:46.290440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3167e-6c2e-4252-977f-e6c3334a217f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:46.291916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d2d3167e-6c2e-4252-977f-e6c3334a217f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:46.304332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3167e-6c2e-4252-977f-e6c3334a217f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:46.308872 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76508440-7afc-4134-bd7c-c35e30038423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:46.309551 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76508440-7afc-4134-bd7c-c35e30038423 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:46.309828 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76508440-7afc-4134-bd7c-c35e30038423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:46.310097 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76508440-7afc-4134-bd7c-c35e30038423 admin admin] PATH_INFO: `/v3/projects/786c60747bf748eab9a907c2ce1d38b6` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:46.311247 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76508440-7afc-4134-bd7c-c35e30038423 admin admin] RBAC: Authorizing `identity:delete_project(project_id=786c60747bf748eab9a907c2ce1d38b6)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:46.313147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76508440-7afc-4134-bd7c-c35e30038423 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:46.360698 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-76508440-7afc-4134-bd7c-c35e30038423 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '786c60747bf748eab9a907c2ce1d38b6', 'request_id': 'req-76508440-7afc-4134-bd7c-c35e30038423'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:46.366879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-76508440-7afc-4134-bd7c-c35e30038423 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '786c60747bf748eab9a907c2ce1d38b6', 'request_id': 'req-76508440-7afc-4134-bd7c-c35e30038423'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:46.375101 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-76508440-7afc-4134-bd7c-c35e30038423 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '786c60747bf748eab9a907c2ce1d38b6', 'request_id': 'req-76508440-7afc-4134-bd7c-c35e30038423'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:46.385639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-76508440-7afc-4134-bd7c-c35e30038423 admin admin] The token cache is being invalidate because project 786c60747bf748eab9a907c2ce1d38b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:35:46.385925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-76508440-7afc-4134-bd7c-c35e30038423 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:46.386571 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-76508440-7afc-4134-bd7c-c35e30038423 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:46.388595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1316/2615] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:35:46 2022] DELETE /identity/v3/projects/786c60747bf748eab9a907c2ce1d38b6 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:46.392556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984581b-6c7b-4e3d-9a67-3362a93011e7 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:46.402012 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d984581b-6c7b-4e3d-9a67-3362a93011e7 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:46.459315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67873b-3e67-4d52-85eb-8f600ac736c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:46.459942 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67873b-3e67-4d52-85eb-8f600ac736c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:46.460219 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67873b-3e67-4d52-85eb-8f600ac736c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:46.460488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c67873b-3e67-4d52-85eb-8f600ac736c3 admin admin] PATH_INFO: `/v3/users/0f01a7a7d33d48fd97f969ab9e80b1e9` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:46.461643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67873b-3e67-4d52-85eb-8f600ac736c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f01a7a7d33d48fd97f969ab9e80b1e9)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:46.463602 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c67873b-3e67-4d52-85eb-8f600ac736c3 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:46.497927 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:46.514297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9c67873b-3e67-4d52-85eb-8f600ac736c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f01a7a7d33d48fd97f969ab9e80b1e9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:46.514946 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9c67873b-3e67-4d52-85eb-8f600ac736c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f01a7a7d33d48fd97f969ab9e80b1e9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:46.527088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9c67873b-3e67-4d52-85eb-8f600ac736c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f01a7a7d33d48fd97f969ab9e80b1e9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:46.535639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9c67873b-3e67-4d52-85eb-8f600ac736c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f01a7a7d33d48fd97f969ab9e80b1e9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:46.536664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9c67873b-3e67-4d52-85eb-8f600ac736c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f01a7a7d33d48fd97f969ab9e80b1e9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:46.556498 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1300/2616] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:35:46 2022] DELETE /identity/v3/users/0f01a7a7d33d48fd97f969ab9e80b1e9 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:46.842755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76508440-7afc-4134-bd7c-c35e30038423 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:46.844183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-76508440-7afc-4134-bd7c-c35e30038423 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:46.860679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76508440-7afc-4134-bd7c-c35e30038423 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:46.920955 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067083b7-c819-45bf-814a-936f5ae85076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:46.921586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067083b7-c819-45bf-814a-936f5ae85076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:46.921897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067083b7-c819-45bf-814a-936f5ae85076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:46.922170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067083b7-c819-45bf-814a-936f5ae85076 admin admin] PATH_INFO: `/v3/projects/06c8d005608647d996f4b277a5344a0e` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:46.923324 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067083b7-c819-45bf-814a-936f5ae85076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06c8d005608647d996f4b277a5344a0e)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:46.925280 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067083b7-c819-45bf-814a-936f5ae85076 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:46.968677 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:46.975888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-067083b7-c819-45bf-814a-936f5ae85076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06c8d005608647d996f4b277a5344a0e', 'request_id': 'req-067083b7-c819-45bf-814a-936f5ae85076'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:46.983510 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-067083b7-c819-45bf-814a-936f5ae85076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06c8d005608647d996f4b277a5344a0e', 'request_id': 'req-067083b7-c819-45bf-814a-936f5ae85076'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:46.991263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-067083b7-c819-45bf-814a-936f5ae85076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06c8d005608647d996f4b277a5344a0e', 'request_id': 'req-067083b7-c819-45bf-814a-936f5ae85076'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:47.003390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-067083b7-c819-45bf-814a-936f5ae85076 admin admin] The token cache is being invalidate because project 06c8d005608647d996f4b277a5344a0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:35:47.003683 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-067083b7-c819-45bf-814a-936f5ae85076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:47.004325 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-067083b7-c819-45bf-814a-936f5ae85076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:47.005830 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1317/2617] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:35:46 2022] DELETE /identity/v3/projects/06c8d005608647d996f4b277a5344a0e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:47.009810 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67873b-3e67-4d52-85eb-8f600ac736c3 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:47.011277 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9c67873b-3e67-4d52-85eb-8f600ac736c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:47.023916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c67873b-3e67-4d52-85eb-8f600ac736c3 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:47.209376 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4739d9-556a-413a-bf24-7e7577bd31fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:47.210012 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4739d9-556a-413a-bf24-7e7577bd31fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:47.210317 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4739d9-556a-413a-bf24-7e7577bd31fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:47.210595 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4739d9-556a-413a-bf24-7e7577bd31fb admin admin] PATH_INFO: `/v3/users/90bf41cb44d946faab66c6830def7cf6` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:47.211715 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4739d9-556a-413a-bf24-7e7577bd31fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=90bf41cb44d946faab66c6830def7cf6)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:47.213656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4739d9-556a-413a-bf24-7e7577bd31fb admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:47.259485 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba4739d9-556a-413a-bf24-7e7577bd31fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90bf41cb44d946faab66c6830def7cf6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:47.260129 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba4739d9-556a-413a-bf24-7e7577bd31fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90bf41cb44d946faab66c6830def7cf6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:47.272144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba4739d9-556a-413a-bf24-7e7577bd31fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90bf41cb44d946faab66c6830def7cf6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:47.279887 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba4739d9-556a-413a-bf24-7e7577bd31fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90bf41cb44d946faab66c6830def7cf6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:47.280527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ba4739d9-556a-413a-bf24-7e7577bd31fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90bf41cb44d946faab66c6830def7cf6'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:47.292893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1301/2618] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:35:47 2022] DELETE /identity/v3/users/90bf41cb44d946faab66c6830def7cf6 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:47.572950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067083b7-c819-45bf-814a-936f5ae85076 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:47.575600 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-067083b7-c819-45bf-814a-936f5ae85076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:47.589343 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067083b7-c819-45bf-814a-936f5ae85076 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:47.655080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5eabfb-3c09-4124-95b6-f7aa726c2cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:47.655719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5eabfb-3c09-4124-95b6-f7aa726c2cdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:47.655998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5eabfb-3c09-4124-95b6-f7aa726c2cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:47.656269 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5eabfb-3c09-4124-95b6-f7aa726c2cdb admin admin] PATH_INFO: `/v3/projects/b44affdacd3b41daaf413a57ad348a93` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:47.657409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5eabfb-3c09-4124-95b6-f7aa726c2cdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=b44affdacd3b41daaf413a57ad348a93)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:47.659711 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5eabfb-3c09-4124-95b6-f7aa726c2cdb admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:47.708790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7d5eabfb-3c09-4124-95b6-f7aa726c2cdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b44affdacd3b41daaf413a57ad348a93', 'request_id': 'req-7d5eabfb-3c09-4124-95b6-f7aa726c2cdb'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:47.714836 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7d5eabfb-3c09-4124-95b6-f7aa726c2cdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b44affdacd3b41daaf413a57ad348a93', 'request_id': 'req-7d5eabfb-3c09-4124-95b6-f7aa726c2cdb'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:47.723719 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7d5eabfb-3c09-4124-95b6-f7aa726c2cdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b44affdacd3b41daaf413a57ad348a93', 'request_id': 'req-7d5eabfb-3c09-4124-95b6-f7aa726c2cdb'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:47.735210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7d5eabfb-3c09-4124-95b6-f7aa726c2cdb admin admin] The token cache is being invalidate because project b44affdacd3b41daaf413a57ad348a93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:35:47.735913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7d5eabfb-3c09-4124-95b6-f7aa726c2cdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:47.736598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-7d5eabfb-3c09-4124-95b6-f7aa726c2cdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:47.738498 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1318/2619] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:35:47 2022] DELETE /identity/v3/projects/b44affdacd3b41daaf413a57ad348a93 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:47.744678 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180d846-59b3-4250-8fd6-9bbb567b82eb None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:47.744901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180d846-59b3-4250-8fd6-9bbb567b82eb None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:47.745172 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1180d846-59b3-4250-8fd6-9bbb567b82eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:47.764823 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-1180d846-59b3-4250-8fd6-9bbb567b82eb None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:47.766013 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-1180d846-59b3-4250-8fd6-9bbb567b82eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:47.787624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:47.830968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1302/2620] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:47.840816 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f583698-e14c-4373-9af0-1007db336691 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:47.841627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f583698-e14c-4373-9af0-1007db336691 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:47.841899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f583698-e14c-4373-9af0-1007db336691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:47.861876 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-0f583698-e14c-4373-9af0-1007db336691 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:47.863115 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0f583698-e14c-4373-9af0-1007db336691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:47.871053 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1319/2621] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:47.875393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180d846-59b3-4250-8fd6-9bbb567b82eb None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:47.884938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1180d846-59b3-4250-8fd6-9bbb567b82eb None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:47.890009 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eee045-f230-4ef8-967d-1d7a95e09889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:47.890633 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eee045-f230-4ef8-967d-1d7a95e09889 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:47.890906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eee045-f230-4ef8-967d-1d7a95e09889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:47.891179 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eee045-f230-4ef8-967d-1d7a95e09889 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:47.891812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eee045-f230-4ef8-967d-1d7a95e09889 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:35:47.892108 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eee045-f230-4ef8-967d-1d7a95e09889 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:47.893665 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eee045-f230-4ef8-967d-1d7a95e09889 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:47.905811 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1303/2622] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:35:47.910524 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f583698-e14c-4373-9af0-1007db336691 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:47.920173 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f583698-e14c-4373-9af0-1007db336691 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:47.925019 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31dc18-fbe3-408d-9ccc-a5a429fb3e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:47.925644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31dc18-fbe3-408d-9ccc-a5a429fb3e5b admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:47.925907 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31dc18-fbe3-408d-9ccc-a5a429fb3e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:47.926177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff31dc18-fbe3-408d-9ccc-a5a429fb3e5b admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:47.926886 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31dc18-fbe3-408d-9ccc-a5a429fb3e5b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:47.928800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff31dc18-fbe3-408d-9ccc-a5a429fb3e5b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:47.956999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1320/2623] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:35:47 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:35:47.961226 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eee045-f230-4ef8-967d-1d7a95e09889 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:47.971197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eee045-f230-4ef8-967d-1d7a95e09889 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:48.030598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4d74a-7740-4af1-8556-37201bde8d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:48.031279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4d74a-7740-4af1-8556-37201bde8d20 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:48.031378 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4d74a-7740-4af1-8556-37201bde8d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:48.031635 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4d74a-7740-4af1-8556-37201bde8d20 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:48.032258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4d74a-7740-4af1-8556-37201bde8d20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:48.034135 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4d74a-7740-4af1-8556-37201bde8d20 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:48.055654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1304/2624] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:35:47 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:35:48.060463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31dc18-fbe3-408d-9ccc-a5a429fb3e5b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:48.070977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff31dc18-fbe3-408d-9ccc-a5a429fb3e5b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:48.076146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa96fd-e7a7-449e-9780-e84396efbb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:48.076738 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa96fd-e7a7-449e-9780-e84396efbb85 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:48.077017 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa96fd-e7a7-449e-9780-e84396efbb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:48.077287 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aa96fd-e7a7-449e-9780-e84396efbb85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:48.077924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa96fd-e7a7-449e-9780-e84396efbb85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:48.079543 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aa96fd-e7a7-449e-9780-e84396efbb85 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:48.089271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1321/2625] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:48.093071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4d74a-7740-4af1-8556-37201bde8d20 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:48.102856 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4d74a-7740-4af1-8556-37201bde8d20 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:48.107628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ba108-31f0-4dec-a235-3f03b88ae56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:48.108218 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ba108-31f0-4dec-a235-3f03b88ae56e admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:48.108495 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ba108-31f0-4dec-a235-3f03b88ae56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:48.108811 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ba108-31f0-4dec-a235-3f03b88ae56e admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:48.109374 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ba108-31f0-4dec-a235-3f03b88ae56e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:48.110974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ba108-31f0-4dec-a235-3f03b88ae56e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:48.121039 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1305/2626] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:48.125740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa96fd-e7a7-449e-9780-e84396efbb85 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:48.135964 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aa96fd-e7a7-449e-9780-e84396efbb85 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:48.141250 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430a48c-f842-4896-acd5-f60abd73c287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:48.141924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430a48c-f842-4896-acd5-f60abd73c287 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:48.142203 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430a48c-f842-4896-acd5-f60abd73c287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:48.142505 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430a48c-f842-4896-acd5-f60abd73c287 admin admin] PATH_INFO: `/v3/projects/3c3fbc2dc9374627bac4054ca68a4064/users/2e57291cbb5c4cae9b1ea7dd3227f289/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:48.156776 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430a48c-f842-4896-acd5-f60abd73c287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c3fbc2dc9374627bac4054ca68a4064, user_id=2e57291cbb5c4cae9b1ea7dd3227f289, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:48.159838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4430a48c-f842-4896-acd5-f60abd73c287 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:48.163089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:48.181603 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1322/2627] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:35:48 2022] PUT /identity/v3/projects/3c3fbc2dc9374627bac4054ca68a4064/users/2e57291cbb5c4cae9b1ea7dd3227f289/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:48.193484 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e6c72-b3ef-48d1-9e5b-98aff37759a4 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:48.193761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e6c72-b3ef-48d1-9e5b-98aff37759a4 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:48.194017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47e6c72-b3ef-48d1-9e5b-98aff37759a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:48.220969 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c47e6c72-b3ef-48d1-9e5b-98aff37759a4 None None] MFA Rules not processed for user `2e57291cbb5c4cae9b1ea7dd3227f289`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:48.222166 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c47e6c72-b3ef-48d1-9e5b-98aff37759a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:48.283931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1306/2628] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:48.288796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430a48c-f842-4896-acd5-f60abd73c287 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:48.357099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4430a48c-f842-4896-acd5-f60abd73c287 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:48.363593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52716950-44cc-4d94-8206-0240b746b438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:48.364199 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52716950-44cc-4d94-8206-0240b746b438 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:48.364481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52716950-44cc-4d94-8206-0240b746b438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:48.364755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52716950-44cc-4d94-8206-0240b746b438 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:48.365385 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52716950-44cc-4d94-8206-0240b746b438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:48.367362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52716950-44cc-4d94-8206-0240b746b438 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:48.397114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1323/2629] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:35:48 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:35:48.401500 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e6c72-b3ef-48d1-9e5b-98aff37759a4 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:48.412547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47e6c72-b3ef-48d1-9e5b-98aff37759a4 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:48.431819 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:48.473580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8be93a-81a0-49a1-b004-c67f4d266b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:48.474181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8be93a-81a0-49a1-b004-c67f4d266b8a admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:48.474488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8be93a-81a0-49a1-b004-c67f4d266b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:48.474775 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8be93a-81a0-49a1-b004-c67f4d266b8a admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:48.475393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8be93a-81a0-49a1-b004-c67f4d266b8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:48.477354 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8be93a-81a0-49a1-b004-c67f4d266b8a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:48.511512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1307/2630] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:35:48 2022] POST /identity/v3/users => generated 300 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:35:48.515552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52716950-44cc-4d94-8206-0240b746b438 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:48.526854 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52716950-44cc-4d94-8206-0240b746b438 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:48.532608 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fe685-060b-46de-9b12-eb13a193ed2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:48.534162 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fe685-060b-46de-9b12-eb13a193ed2c admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:48.534695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fe685-060b-46de-9b12-eb13a193ed2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:48.534972 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47fe685-060b-46de-9b12-eb13a193ed2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:48.535620 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fe685-060b-46de-9b12-eb13a193ed2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:48.537308 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47fe685-060b-46de-9b12-eb13a193ed2c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:48.549032 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1324/2631] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:48.553489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8be93a-81a0-49a1-b004-c67f4d266b8a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:48.563514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8be93a-81a0-49a1-b004-c67f4d266b8a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:48.569197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc6fd3-9e6f-4378-97d5-9b8ea3e1064d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:48.570007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc6fd3-9e6f-4378-97d5-9b8ea3e1064d admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:48.570323 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc6fd3-9e6f-4378-97d5-9b8ea3e1064d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:48.570697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc6fd3-9e6f-4378-97d5-9b8ea3e1064d admin admin] PATH_INFO: `/v3/projects/05b441f240094383a657313853c17b7f/users/6355b8d812f04e768657655f2a9c3497/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:48.588450 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc6fd3-9e6f-4378-97d5-9b8ea3e1064d admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b441f240094383a657313853c17b7f, user_id=6355b8d812f04e768657655f2a9c3497, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:48.591931 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc6fd3-9e6f-4378-97d5-9b8ea3e1064d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:48.605451 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1308/2632] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:35:48 2022] PUT /identity/v3/projects/05b441f240094383a657313853c17b7f/users/6355b8d812f04e768657655f2a9c3497/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:48.609473 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fe685-060b-46de-9b12-eb13a193ed2c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:48.620319 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47fe685-060b-46de-9b12-eb13a193ed2c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:48.680606 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f20c36-5957-4814-85ee-9f9924ff31ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:48.681212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f20c36-5957-4814-85ee-9f9924ff31ed admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:48.681487 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f20c36-5957-4814-85ee-9f9924ff31ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:48.681753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f20c36-5957-4814-85ee-9f9924ff31ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:48.682626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f20c36-5957-4814-85ee-9f9924ff31ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:48.684991 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f20c36-5957-4814-85ee-9f9924ff31ed admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:48.702595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1325/2633] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:48 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:48.705430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc6fd3-9e6f-4378-97d5-9b8ea3e1064d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:48.716660 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc6fd3-9e6f-4378-97d5-9b8ea3e1064d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:48.722145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b956e8-fe1f-4b20-acee-71c562376c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:48.723074 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b956e8-fe1f-4b20-acee-71c562376c5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:48.723547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b956e8-fe1f-4b20-acee-71c562376c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:48.723829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b956e8-fe1f-4b20-acee-71c562376c5a admin admin] PATH_INFO: `/v3/projects/05b441f240094383a657313853c17b7f/users/6355b8d812f04e768657655f2a9c3497/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:48.727469 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b956e8-fe1f-4b20-acee-71c562376c5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b441f240094383a657313853c17b7f, user_id=6355b8d812f04e768657655f2a9c3497, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:48.730245 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b956e8-fe1f-4b20-acee-71c562376c5a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:48.742514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1309/2634] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:35:48 2022] PUT /identity/v3/projects/05b441f240094383a657313853c17b7f/users/6355b8d812f04e768657655f2a9c3497/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:48.753691 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca05b29-b4e1-4baa-82ab-8fa8e66e6b1e None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:48.753974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca05b29-b4e1-4baa-82ab-8fa8e66e6b1e None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:48.754271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca05b29-b4e1-4baa-82ab-8fa8e66e6b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:48.784011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-aca05b29-b4e1-4baa-82ab-8fa8e66e6b1e None None] MFA Rules not processed for user `6355b8d812f04e768657655f2a9c3497`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:48.785544 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-aca05b29-b4e1-4baa-82ab-8fa8e66e6b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:48.840316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:48.858905 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1326/2635] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:35:48.871410 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b956e8-fe1f-4b20-acee-71c562376c5a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:48.873656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-63b956e8-fe1f-4b20-acee-71c562376c5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:48.932385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b956e8-fe1f-4b20-acee-71c562376c5a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:48.936549 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d0e0b-44d8-4f4e-a791-75b73eefc339 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:48.937154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d0e0b-44d8-4f4e-a791-75b73eefc339 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:48.937429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d0e0b-44d8-4f4e-a791-75b73eefc339 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:48.937698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d0e0b-44d8-4f4e-a791-75b73eefc339 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:48.946483 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d0e0b-44d8-4f4e-a791-75b73eefc339 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:48.947893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7d0e0b-44d8-4f4e-a791-75b73eefc339 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:48.958092 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1310/2636] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:49.510386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca05b29-b4e1-4baa-82ab-8fa8e66e6b1e None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:49.573085 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca05b29-b4e1-4baa-82ab-8fa8e66e6b1e None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:49.577796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdc8ea-3f4d-4324-9cf0-196f76df0c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:49.578397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdc8ea-3f4d-4324-9cf0-196f76df0c57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:49.578674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdc8ea-3f4d-4324-9cf0-196f76df0c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:49.578943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacdc8ea-3f4d-4324-9cf0-196f76df0c57 admin admin] PATH_INFO: `/v3/users/2e57291cbb5c4cae9b1ea7dd3227f289` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:49.580159 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacdc8ea-3f4d-4324-9cf0-196f76df0c57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e57291cbb5c4cae9b1ea7dd3227f289)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:49.582102 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacdc8ea-3f4d-4324-9cf0-196f76df0c57 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:49.630559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dacdc8ea-3f4d-4324-9cf0-196f76df0c57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e57291cbb5c4cae9b1ea7dd3227f289'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:49.638106 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dacdc8ea-3f4d-4324-9cf0-196f76df0c57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e57291cbb5c4cae9b1ea7dd3227f289'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:49.649956 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dacdc8ea-3f4d-4324-9cf0-196f76df0c57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e57291cbb5c4cae9b1ea7dd3227f289'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:49.650586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dacdc8ea-3f4d-4324-9cf0-196f76df0c57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e57291cbb5c4cae9b1ea7dd3227f289'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:49.658126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dacdc8ea-3f4d-4324-9cf0-196f76df0c57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e57291cbb5c4cae9b1ea7dd3227f289'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:49.664082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1327/2637] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:35:49 2022] DELETE /identity/v3/users/2e57291cbb5c4cae9b1ea7dd3227f289 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:49.668501 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a714e51-896a-4350-b1d0-0660cc25df11 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:49.669582 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a714e51-896a-4350-b1d0-0660cc25df11 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:49.669682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a714e51-896a-4350-b1d0-0660cc25df11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:49.687796 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-6a714e51-896a-4350-b1d0-0660cc25df11 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:49.688992 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6a714e51-896a-4350-b1d0-0660cc25df11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:49.749242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1311/2638] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:49.759110 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdc8ea-3f4d-4324-9cf0-196f76df0c57 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:49.760569 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-dacdc8ea-3f4d-4324-9cf0-196f76df0c57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:49.821043 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacdc8ea-3f4d-4324-9cf0-196f76df0c57 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:49.824692 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b4059-ba49-44d6-914b-aea749d8e0f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:49.825820 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b4059-ba49-44d6-914b-aea749d8e0f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:49.826117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b4059-ba49-44d6-914b-aea749d8e0f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:49.826420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907b4059-ba49-44d6-914b-aea749d8e0f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:49.836000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b4059-ba49-44d6-914b-aea749d8e0f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:49.837475 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907b4059-ba49-44d6-914b-aea749d8e0f1 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:49.847769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1328/2639] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:50.124685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a714e51-896a-4350-b1d0-0660cc25df11 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:50.126207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6a714e51-896a-4350-b1d0-0660cc25df11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:50.139487 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a714e51-896a-4350-b1d0-0660cc25df11 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:50.144156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c893573-36cb-4115-b3a3-c75cb303f52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:50.145201 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c893573-36cb-4115-b3a3-c75cb303f52b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:50.145201 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c893573-36cb-4115-b3a3-c75cb303f52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:50.145415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c893573-36cb-4115-b3a3-c75cb303f52b admin admin] PATH_INFO: `/v3/projects/3c3fbc2dc9374627bac4054ca68a4064` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:50.146466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c893573-36cb-4115-b3a3-c75cb303f52b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c3fbc2dc9374627bac4054ca68a4064)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:50.148351 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c893573-36cb-4115-b3a3-c75cb303f52b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:50.161024 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:50.196639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c893573-36cb-4115-b3a3-c75cb303f52b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c3fbc2dc9374627bac4054ca68a4064', 'request_id': 'req-5c893573-36cb-4115-b3a3-c75cb303f52b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:50.202187 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c893573-36cb-4115-b3a3-c75cb303f52b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c3fbc2dc9374627bac4054ca68a4064', 'request_id': 'req-5c893573-36cb-4115-b3a3-c75cb303f52b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:50.209590 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c893573-36cb-4115-b3a3-c75cb303f52b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c3fbc2dc9374627bac4054ca68a4064', 'request_id': 'req-5c893573-36cb-4115-b3a3-c75cb303f52b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:50.221165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c893573-36cb-4115-b3a3-c75cb303f52b admin admin] The token cache is being invalidate because project 3c3fbc2dc9374627bac4054ca68a4064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:35:50.221456 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c893573-36cb-4115-b3a3-c75cb303f52b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:50.222172 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5c893573-36cb-4115-b3a3-c75cb303f52b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:50.223679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1312/2640] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:35:50 2022] DELETE /identity/v3/projects/3c3fbc2dc9374627bac4054ca68a4064 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:50.227907 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b4059-ba49-44d6-914b-aea749d8e0f1 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:50.238459 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907b4059-ba49-44d6-914b-aea749d8e0f1 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:50.247701 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:50.298576 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73267c-1e73-4c73-8fde-ad9f04ba8eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:50.299394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73267c-1e73-4c73-8fde-ad9f04ba8eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:50.299497 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73267c-1e73-4c73-8fde-ad9f04ba8eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:50.299709 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73267c-1e73-4c73-8fde-ad9f04ba8eb9 admin admin] PATH_INFO: `/v3/users/6355b8d812f04e768657655f2a9c3497` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:50.300843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73267c-1e73-4c73-8fde-ad9f04ba8eb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6355b8d812f04e768657655f2a9c3497)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:50.302783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73267c-1e73-4c73-8fde-ad9f04ba8eb9 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:50.350397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-be73267c-1e73-4c73-8fde-ad9f04ba8eb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6355b8d812f04e768657655f2a9c3497'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:50.358881 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-be73267c-1e73-4c73-8fde-ad9f04ba8eb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6355b8d812f04e768657655f2a9c3497'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:50.370800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-be73267c-1e73-4c73-8fde-ad9f04ba8eb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6355b8d812f04e768657655f2a9c3497'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:50.371465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-be73267c-1e73-4c73-8fde-ad9f04ba8eb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6355b8d812f04e768657655f2a9c3497'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:50.378322 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-be73267c-1e73-4c73-8fde-ad9f04ba8eb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6355b8d812f04e768657655f2a9c3497'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:50.384856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1329/2641] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:35:50 2022] DELETE /identity/v3/users/6355b8d812f04e768657655f2a9c3497 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:50.770558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c893573-36cb-4115-b3a3-c75cb303f52b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:50.772009 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5c893573-36cb-4115-b3a3-c75cb303f52b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:50.785131 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c893573-36cb-4115-b3a3-c75cb303f52b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:50.844739 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c66920d-21ff-465d-9d69-722c6f83e619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:50.844739 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66920d-21ff-465d-9d69-722c6f83e619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:50.844739 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66920d-21ff-465d-9d69-722c6f83e619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:50.844739 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c66920d-21ff-465d-9d69-722c6f83e619 admin admin] PATH_INFO: `/v3/projects/05b441f240094383a657313853c17b7f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:50.845573 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66920d-21ff-465d-9d69-722c6f83e619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05b441f240094383a657313853c17b7f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:50.847596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c66920d-21ff-465d-9d69-722c6f83e619 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:50.893704 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3c66920d-21ff-465d-9d69-722c6f83e619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05b441f240094383a657313853c17b7f', 'request_id': 'req-3c66920d-21ff-465d-9d69-722c6f83e619'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:50.899225 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3c66920d-21ff-465d-9d69-722c6f83e619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05b441f240094383a657313853c17b7f', 'request_id': 'req-3c66920d-21ff-465d-9d69-722c6f83e619'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:50.907317 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3c66920d-21ff-465d-9d69-722c6f83e619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05b441f240094383a657313853c17b7f', 'request_id': 'req-3c66920d-21ff-465d-9d69-722c6f83e619'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:50.919514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3c66920d-21ff-465d-9d69-722c6f83e619 admin admin] The token cache is being invalidate because project 05b441f240094383a657313853c17b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:35:50.919801 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3c66920d-21ff-465d-9d69-722c6f83e619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:50.920432 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-3c66920d-21ff-465d-9d69-722c6f83e619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:35:50.921891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1313/2642] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:35:50 2022] DELETE /identity/v3/projects/05b441f240094383a657313853c17b7f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:50.926871 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abe323-5bd5-4a37-9b80-e4dd9114f59b None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:50.927344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abe323-5bd5-4a37-9b80-e4dd9114f59b None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:50.927453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43abe323-5bd5-4a37-9b80-e4dd9114f59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:50.947838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-43abe323-5bd5-4a37-9b80-e4dd9114f59b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:50.949046 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-43abe323-5bd5-4a37-9b80-e4dd9114f59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:51.012688 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1330/2643] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:51.076151 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c61f8-9b46-4a9c-90c9-b0fdd852f7e2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:51.076435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c61f8-9b46-4a9c-90c9-b0fdd852f7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:51.076711 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927c61f8-9b46-4a9c-90c9-b0fdd852f7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:51.096837 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-927c61f8-9b46-4a9c-90c9-b0fdd852f7e2 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:51.098043 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-927c61f8-9b46-4a9c-90c9-b0fdd852f7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:51.105912 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1314/2644] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:51.111018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43abe323-5bd5-4a37-9b80-e4dd9114f59b None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:51.121047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43abe323-5bd5-4a37-9b80-e4dd9114f59b None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:51.125754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e3d54-3ac7-4956-86c6-afd775a1763c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:51.126515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e3d54-3ac7-4956-86c6-afd775a1763c admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:51.126933 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e3d54-3ac7-4956-86c6-afd775a1763c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:51.127209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1e3d54-3ac7-4956-86c6-afd775a1763c admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:51.127850 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e3d54-3ac7-4956-86c6-afd775a1763c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:35:51.128149 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e3d54-3ac7-4956-86c6-afd775a1763c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:51.129767 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1e3d54-3ac7-4956-86c6-afd775a1763c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:51.141968 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1331/2645] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:35:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:35:51.146823 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c61f8-9b46-4a9c-90c9-b0fdd852f7e2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:51.156505 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927c61f8-9b46-4a9c-90c9-b0fdd852f7e2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:51.161337 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644079d6-6500-4baf-a435-96295ea9d387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:51.161948 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644079d6-6500-4baf-a435-96295ea9d387 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:51.162234 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644079d6-6500-4baf-a435-96295ea9d387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:51.162538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644079d6-6500-4baf-a435-96295ea9d387 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:51.163150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644079d6-6500-4baf-a435-96295ea9d387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:51.165005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644079d6-6500-4baf-a435-96295ea9d387 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:51.192197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1315/2646] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:35:51 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:35:51.196309 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e3d54-3ac7-4956-86c6-afd775a1763c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:51.206208 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1e3d54-3ac7-4956-86c6-afd775a1763c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:51.223332 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:51.265146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a54878-4127-42c6-85e7-5e99f87c5f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:51.265744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a54878-4127-42c6-85e7-5e99f87c5f2e admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:51.266020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a54878-4127-42c6-85e7-5e99f87c5f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:51.266316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a54878-4127-42c6-85e7-5e99f87c5f2e admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:51.266986 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a54878-4127-42c6-85e7-5e99f87c5f2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:51.268891 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a54878-4127-42c6-85e7-5e99f87c5f2e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:51.290723 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1332/2647] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:35:51 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:35:51.295488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644079d6-6500-4baf-a435-96295ea9d387 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:51.301200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:51.307596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644079d6-6500-4baf-a435-96295ea9d387 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:51.313488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea41dc5-5837-4a10-8aff-c121f3bea4dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:51.314111 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea41dc5-5837-4a10-8aff-c121f3bea4dd admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:51.314415 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea41dc5-5837-4a10-8aff-c121f3bea4dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:51.314682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea41dc5-5837-4a10-8aff-c121f3bea4dd admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:51.315359 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea41dc5-5837-4a10-8aff-c121f3bea4dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:51.316981 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea41dc5-5837-4a10-8aff-c121f3bea4dd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:51.327311 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1316/2648] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:51.331268 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a54878-4127-42c6-85e7-5e99f87c5f2e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:51.342059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a54878-4127-42c6-85e7-5e99f87c5f2e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:51.347065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19e5dc9-f51b-4feb-81c6-ef2a5920e8d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:51.347668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e5dc9-f51b-4feb-81c6-ef2a5920e8d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:51.347949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e5dc9-f51b-4feb-81c6-ef2a5920e8d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:51.348218 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19e5dc9-f51b-4feb-81c6-ef2a5920e8d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:51.348856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e5dc9-f51b-4feb-81c6-ef2a5920e8d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:51.350508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19e5dc9-f51b-4feb-81c6-ef2a5920e8d6 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:51.361114 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1333/2649] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:51.365128 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea41dc5-5837-4a10-8aff-c121f3bea4dd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:51.375246 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea41dc5-5837-4a10-8aff-c121f3bea4dd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:51.380495 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c905c1-2600-4f1b-8a3e-6423d3e7e415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:51.384741 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c905c1-2600-4f1b-8a3e-6423d3e7e415 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:51.384741 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c905c1-2600-4f1b-8a3e-6423d3e7e415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:51.384741 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c905c1-2600-4f1b-8a3e-6423d3e7e415 admin admin] PATH_INFO: `/v3/projects/42837871e0684aeaaf26b56202192668/users/c48977da1d8647699e30386818392246/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:51.397048 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c905c1-2600-4f1b-8a3e-6423d3e7e415 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42837871e0684aeaaf26b56202192668, user_id=c48977da1d8647699e30386818392246, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:51.399615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c905c1-2600-4f1b-8a3e-6423d3e7e415 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:51.413094 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1317/2650] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:35:51 2022] PUT /identity/v3/projects/42837871e0684aeaaf26b56202192668/users/c48977da1d8647699e30386818392246/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:51.418175 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b92a5d-4adf-4070-89fd-1a793d823d55 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:51.418494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b92a5d-4adf-4070-89fd-1a793d823d55 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:51.418767 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b92a5d-4adf-4070-89fd-1a793d823d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:51.437637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e7b92a5d-4adf-4070-89fd-1a793d823d55 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:51.438878 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e7b92a5d-4adf-4070-89fd-1a793d823d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:51.499266 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1334/2651] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:51.508985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c905c1-2600-4f1b-8a3e-6423d3e7e415 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:51.510768 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-57c905c1-2600-4f1b-8a3e-6423d3e7e415 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:51.567791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c905c1-2600-4f1b-8a3e-6423d3e7e415 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:51.571501 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf311d4-8a8c-48ed-ab13-3d30775bab5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:51.572094 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf311d4-8a8c-48ed-ab13-3d30775bab5c service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:51.572367 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf311d4-8a8c-48ed-ab13-3d30775bab5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:51.572632 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf311d4-8a8c-48ed-ab13-3d30775bab5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:51.581431 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf311d4-8a8c-48ed-ab13-3d30775bab5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:51.582869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf311d4-8a8c-48ed-ab13-3d30775bab5c service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:51.591955 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1318/2652] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:52.279316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfd317-8053-4a2e-899e-2fcf6cdb6d86 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:52.279854 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfd317-8053-4a2e-899e-2fcf6cdb6d86 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:52.279854 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfd317-8053-4a2e-899e-2fcf6cdb6d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:52.320961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-72bfd317-8053-4a2e-899e-2fcf6cdb6d86 None None] MFA Rules not processed for user `c48977da1d8647699e30386818392246`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:52.322427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-72bfd317-8053-4a2e-899e-2fcf6cdb6d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:52.412104 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1335/2653] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:52.420233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf311d4-8a8c-48ed-ab13-3d30775bab5c service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:52.426791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf311d4-8a8c-48ed-ab13-3d30775bab5c service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:52.431467 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907cc61f-c2d9-4a4f-bfe8-e3e2cd1336e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:52.432079 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cc61f-c2d9-4a4f-bfe8-e3e2cd1336e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:52.432338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cc61f-c2d9-4a4f-bfe8-e3e2cd1336e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:52.432608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907cc61f-c2d9-4a4f-bfe8-e3e2cd1336e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:52.433223 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907cc61f-c2d9-4a4f-bfe8-e3e2cd1336e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:52.435690 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907cc61f-c2d9-4a4f-bfe8-e3e2cd1336e8 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:52.465455 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1319/2654] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:35:52 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:35:52.469566 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfd317-8053-4a2e-899e-2fcf6cdb6d86 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:52.481731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfd317-8053-4a2e-899e-2fcf6cdb6d86 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:52.509001 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:52.541299 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca8f06-b0db-4a4b-9998-05bdb6781d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:52.541916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca8f06-b0db-4a4b-9998-05bdb6781d5e admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:52.542233 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca8f06-b0db-4a4b-9998-05bdb6781d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:52.542493 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca8f06-b0db-4a4b-9998-05bdb6781d5e admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:52.543119 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca8f06-b0db-4a4b-9998-05bdb6781d5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:52.545064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca8f06-b0db-4a4b-9998-05bdb6781d5e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:52.569212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1336/2655] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:35:52 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:35:52.570403 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907cc61f-c2d9-4a4f-bfe8-e3e2cd1336e8 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:52.581272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907cc61f-c2d9-4a4f-bfe8-e3e2cd1336e8 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:52.585775 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90523216-6615-48dc-a1f9-7392fe548d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:52.586421 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90523216-6615-48dc-a1f9-7392fe548d41 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:52.586701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90523216-6615-48dc-a1f9-7392fe548d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:52.586973 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90523216-6615-48dc-a1f9-7392fe548d41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:52.587608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90523216-6615-48dc-a1f9-7392fe548d41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:52.589183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90523216-6615-48dc-a1f9-7392fe548d41 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:52.599672 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1320/2656] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:52.603769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca8f06-b0db-4a4b-9998-05bdb6781d5e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:52.614107 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca8f06-b0db-4a4b-9998-05bdb6781d5e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:52.618912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94848e08-8cb2-421b-b381-f2f2bb125ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:52.619559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94848e08-8cb2-421b-b381-f2f2bb125ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:52.619840 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94848e08-8cb2-421b-b381-f2f2bb125ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:52.620117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94848e08-8cb2-421b-b381-f2f2bb125ea3 admin admin] PATH_INFO: `/v3/projects/8e55ff60340f48558375780f4887a9a9/users/b5ba12ef6b3f465ca092491ee01f3c60/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:52.636094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94848e08-8cb2-421b-b381-f2f2bb125ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e55ff60340f48558375780f4887a9a9, user_id=b5ba12ef6b3f465ca092491ee01f3c60, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:52.639970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94848e08-8cb2-421b-b381-f2f2bb125ea3 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:52.652687 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1337/2657] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:35:52 2022] PUT /identity/v3/projects/8e55ff60340f48558375780f4887a9a9/users/b5ba12ef6b3f465ca092491ee01f3c60/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:52.656731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90523216-6615-48dc-a1f9-7392fe548d41 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:52.667337 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90523216-6615-48dc-a1f9-7392fe548d41 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:52.722830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce6861-f377-4181-9940-d2fde33c2415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:52.723506 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce6861-f377-4181-9940-d2fde33c2415 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:52.723780 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce6861-f377-4181-9940-d2fde33c2415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:52.724046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce6861-f377-4181-9940-d2fde33c2415 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:52.724669 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce6861-f377-4181-9940-d2fde33c2415 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:52.726250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce6861-f377-4181-9940-d2fde33c2415 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:52.736492 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1321/2658] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:35:52 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:35:52.740974 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94848e08-8cb2-421b-b381-f2f2bb125ea3 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:52.751754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94848e08-8cb2-421b-b381-f2f2bb125ea3 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:52.756410 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6aabb9-6a19-403c-b3df-c0834c591303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:52.757050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6aabb9-6a19-403c-b3df-c0834c591303 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:52.757326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6aabb9-6a19-403c-b3df-c0834c591303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:52.757596 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6aabb9-6a19-403c-b3df-c0834c591303 admin admin] PATH_INFO: `/v3/projects/8e55ff60340f48558375780f4887a9a9/users/b5ba12ef6b3f465ca092491ee01f3c60/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:52.759583 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6aabb9-6a19-403c-b3df-c0834c591303 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e55ff60340f48558375780f4887a9a9, user_id=b5ba12ef6b3f465ca092491ee01f3c60, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:52.762199 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6aabb9-6a19-403c-b3df-c0834c591303 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:52.772699 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1338/2659] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:35:52 2022] PUT /identity/v3/projects/8e55ff60340f48558375780f4887a9a9/users/b5ba12ef6b3f465ca092491ee01f3c60/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:35:53.417808 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83885a57-45a8-4759-b33e-245e388b8f4a None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:53.418069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83885a57-45a8-4759-b33e-245e388b8f4a None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:53.418372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83885a57-45a8-4759-b33e-245e388b8f4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:53.449312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-83885a57-45a8-4759-b33e-245e388b8f4a None None] MFA Rules not processed for user `b5ba12ef6b3f465ca092491ee01f3c60`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:35:53.451234 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-83885a57-45a8-4759-b33e-245e388b8f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:53.476588 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:35:53.555001 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1322/2660] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:35:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:35:53.566460 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6aabb9-6a19-403c-b3df-c0834c591303 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:53.569587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bf6aabb9-6a19-403c-b3df-c0834c591303 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:53.638922 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6aabb9-6a19-403c-b3df-c0834c591303 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:53.642855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea990611-229f-4ec9-9ad2-2f02f59961bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:53.643463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea990611-229f-4ec9-9ad2-2f02f59961bf service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:53.643736 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea990611-229f-4ec9-9ad2-2f02f59961bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:53.644011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea990611-229f-4ec9-9ad2-2f02f59961bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:53.700523 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea990611-229f-4ec9-9ad2-2f02f59961bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:53.701971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea990611-229f-4ec9-9ad2-2f02f59961bf service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:53.711109 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1339/2661] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:35:55.086047 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83885a57-45a8-4759-b33e-245e388b8f4a None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:35:55.087458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-83885a57-45a8-4759-b33e-245e388b8f4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:35:55.100289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83885a57-45a8-4759-b33e-245e388b8f4a None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:35:55.103898 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4a69cb-974f-4021-a582-2c3fe1e87851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:35:55.104498 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a69cb-974f-4021-a582-2c3fe1e87851 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:35:55.104772 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a69cb-974f-4021-a582-2c3fe1e87851 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:35:55.105038 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4a69cb-974f-4021-a582-2c3fe1e87851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:35:55.114579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a69cb-974f-4021-a582-2c3fe1e87851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:35:55.116011 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4a69cb-974f-4021-a582-2c3fe1e87851 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:35:55.125986 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1323/2662] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:35:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:36:03.706887 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:05.122003 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=25 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:06.787375 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff9d76-bbca-4d03-98a7-121d84790546 None None] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:06.787942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff9d76-bbca-4d03-98a7-121d84790546 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:06.788293 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ff9d76-bbca-4d03-98a7-121d84790546 None None] PATH_INFO: `/` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:06.789702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1340/2663] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 15:36:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:36:06.796708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baa369-3ce8-4616-939c-83ac81e06ce2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:06.797197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baa369-3ce8-4616-939c-83ac81e06ce2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:06.797315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baa369-3ce8-4616-939c-83ac81e06ce2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:06.820451 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-54baa369-3ce8-4616-939c-83ac81e06ce2 None None] MFA Rules not processed for user `5427bd0ac88042338b956c10cfd07baa`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:36:06.821878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-54baa369-3ce8-4616-939c-83ac81e06ce2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:06.887275 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1324/2664] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 15:36:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:36:06.894655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff9d76-bbca-4d03-98a7-121d84790546 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:06.897025 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b1ff9d76-bbca-4d03-98a7-121d84790546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:06.911463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ff9d76-bbca-4d03-98a7-121d84790546 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:06.915092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608a5e8-32d2-42b0-ba55-7a951b91d57f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:06.915706 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608a5e8-32d2-42b0-ba55-7a951b91d57f service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:06.915983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608a5e8-32d2-42b0-ba55-7a951b91d57f service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:06.916273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608a5e8-32d2-42b0-ba55-7a951b91d57f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:06.925028 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608a5e8-32d2-42b0-ba55-7a951b91d57f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:06.926611 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608a5e8-32d2-42b0-ba55-7a951b91d57f service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:06.936220 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1341/2665] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:36:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:36:07.023725 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c9240-26c2-4c9d-8451-4ec54d5e84de None None] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:07.024127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c9240-26c2-4c9d-8451-4ec54d5e84de None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:07.024493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9c9240-26c2-4c9d-8451-4ec54d5e84de None None] PATH_INFO: `/` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:07.025909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1325/2666] 10.222.0.76 () {54 vars in 921 bytes} [Wed Aug 24 15:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 15:36:07.031971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eab10b-1d58-4749-9f2c-23b8e6bd8663 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:07.032326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eab10b-1d58-4749-9f2c-23b8e6bd8663 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:07.032875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eab10b-1d58-4749-9f2c-23b8e6bd8663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:07.060325 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a6eab10b-1d58-4749-9f2c-23b8e6bd8663 None None] MFA Rules not processed for user `8a5fd31a2db944f596912e4f0aa26835`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:36:07.060325 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a6eab10b-1d58-4749-9f2c-23b8e6bd8663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:07.136136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1342/2667] 10.222.0.76 () {58 vars in 1018 bytes} [Wed Aug 24 15:36:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:36:07.148436 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c9240-26c2-4c9d-8451-4ec54d5e84de None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:07.161055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9c9240-26c2-4c9d-8451-4ec54d5e84de None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:07.164670 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61fa60-0621-49ec-a996-ae232effd5cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:07.165315 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61fa60-0621-49ec-a996-ae232effd5cf service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:07.165590 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61fa60-0621-49ec-a996-ae232effd5cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:07.165879 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61fa60-0621-49ec-a996-ae232effd5cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:07.176923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61fa60-0621-49ec-a996-ae232effd5cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:07.178453 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61fa60-0621-49ec-a996-ae232effd5cf service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:07.189039 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1326/2668] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:36:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:36:08.307726 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cd672-4b94-4a25-93ba-460de6cb54af None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:08.308198 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cd672-4b94-4a25-93ba-460de6cb54af None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:08.308331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641cd672-4b94-4a25-93ba-460de6cb54af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:08.336704 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-641cd672-4b94-4a25-93ba-460de6cb54af None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:36:08.338011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-641cd672-4b94-4a25-93ba-460de6cb54af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:08.398581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1343/2669] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 15:36:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:36:08.447601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61fa60-0621-49ec-a996-ae232effd5cf service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:08.449019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-ad61fa60-0621-49ec-a996-ae232effd5cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:08.528661 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61fa60-0621-49ec-a996-ae232effd5cf service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:08.538476 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a19231-98ad-4ea1-84b7-57e052d5194d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:08.539095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a19231-98ad-4ea1-84b7-57e052d5194d service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:08.539369 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a19231-98ad-4ea1-84b7-57e052d5194d service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:08.539745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a19231-98ad-4ea1-84b7-57e052d5194d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:08.549209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a19231-98ad-4ea1-84b7-57e052d5194d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:08.551115 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a19231-98ad-4ea1-84b7-57e052d5194d service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:08.561819 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1327/2670] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:36:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:36:10.399095 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641cd672-4b94-4a25-93ba-460de6cb54af None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:10.486733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641cd672-4b94-4a25-93ba-460de6cb54af None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:10.491252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302091a-5b3f-4b25-9c9f-77d17bb9745b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:10.491882 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302091a-5b3f-4b25-9c9f-77d17bb9745b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:10.492160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302091a-5b3f-4b25-9c9f-77d17bb9745b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:10.492429 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0302091a-5b3f-4b25-9c9f-77d17bb9745b admin admin] PATH_INFO: `/v3/users/c48977da1d8647699e30386818392246` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:10.493559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302091a-5b3f-4b25-9c9f-77d17bb9745b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c48977da1d8647699e30386818392246)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:10.495522 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0302091a-5b3f-4b25-9c9f-77d17bb9745b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:10.543852 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:10.570598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0302091a-5b3f-4b25-9c9f-77d17bb9745b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c48977da1d8647699e30386818392246'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:10.584323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0302091a-5b3f-4b25-9c9f-77d17bb9745b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c48977da1d8647699e30386818392246'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:10.601931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0302091a-5b3f-4b25-9c9f-77d17bb9745b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c48977da1d8647699e30386818392246'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:10.602788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0302091a-5b3f-4b25-9c9f-77d17bb9745b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c48977da1d8647699e30386818392246'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:10.611815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-0302091a-5b3f-4b25-9c9f-77d17bb9745b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c48977da1d8647699e30386818392246'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:10.618641 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1344/2671] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:36:10 2022] DELETE /identity/v3/users/c48977da1d8647699e30386818392246 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:36:10.859670 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a19231-98ad-4ea1-84b7-57e052d5194d service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:10.878543 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a19231-98ad-4ea1-84b7-57e052d5194d service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:11.002447 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed0485-bc1c-489c-93f6-cd88325f3f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:11.004016 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed0485-bc1c-489c-93f6-cd88325f3f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:11.004459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed0485-bc1c-489c-93f6-cd88325f3f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:11.004918 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbed0485-bc1c-489c-93f6-cd88325f3f29 admin admin] PATH_INFO: `/v3/projects/42837871e0684aeaaf26b56202192668` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:11.006506 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed0485-bc1c-489c-93f6-cd88325f3f29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42837871e0684aeaaf26b56202192668)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:11.020916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbed0485-bc1c-489c-93f6-cd88325f3f29 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:11.095642 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:11.138057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cbed0485-bc1c-489c-93f6-cd88325f3f29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42837871e0684aeaaf26b56202192668', 'request_id': 'req-cbed0485-bc1c-489c-93f6-cd88325f3f29'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:11.145613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cbed0485-bc1c-489c-93f6-cd88325f3f29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42837871e0684aeaaf26b56202192668', 'request_id': 'req-cbed0485-bc1c-489c-93f6-cd88325f3f29'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:11.162850 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cbed0485-bc1c-489c-93f6-cd88325f3f29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42837871e0684aeaaf26b56202192668', 'request_id': 'req-cbed0485-bc1c-489c-93f6-cd88325f3f29'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:11.180271 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cbed0485-bc1c-489c-93f6-cd88325f3f29 admin admin] The token cache is being invalidate because project 42837871e0684aeaaf26b56202192668 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:36:11.180813 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cbed0485-bc1c-489c-93f6-cd88325f3f29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:11.181620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-cbed0485-bc1c-489c-93f6-cd88325f3f29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:11.185247 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1328/2672] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:36:10 2022] DELETE /identity/v3/projects/42837871e0684aeaaf26b56202192668 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:36:11.191663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302091a-5b3f-4b25-9c9f-77d17bb9745b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:11.193400 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0302091a-5b3f-4b25-9c9f-77d17bb9745b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:11.212910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0302091a-5b3f-4b25-9c9f-77d17bb9745b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:11.302190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2094666-4a03-4fee-b30f-8fd1940bdfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:11.302923 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2094666-4a03-4fee-b30f-8fd1940bdfaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:11.303274 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2094666-4a03-4fee-b30f-8fd1940bdfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:11.303717 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2094666-4a03-4fee-b30f-8fd1940bdfaa admin admin] PATH_INFO: `/v3/users/b5ba12ef6b3f465ca092491ee01f3c60` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:11.305189 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2094666-4a03-4fee-b30f-8fd1940bdfaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5ba12ef6b3f465ca092491ee01f3c60)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:11.307287 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2094666-4a03-4fee-b30f-8fd1940bdfaa admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:11.374853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a2094666-4a03-4fee-b30f-8fd1940bdfaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5ba12ef6b3f465ca092491ee01f3c60'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:11.389231 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a2094666-4a03-4fee-b30f-8fd1940bdfaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5ba12ef6b3f465ca092491ee01f3c60'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:11.404514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a2094666-4a03-4fee-b30f-8fd1940bdfaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5ba12ef6b3f465ca092491ee01f3c60'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:11.406096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a2094666-4a03-4fee-b30f-8fd1940bdfaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5ba12ef6b3f465ca092491ee01f3c60'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:11.429532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-a2094666-4a03-4fee-b30f-8fd1940bdfaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5ba12ef6b3f465ca092491ee01f3c60'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:11.437000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1345/2673] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:36:11 2022] DELETE /identity/v3/users/b5ba12ef6b3f465ca092491ee01f3c60 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:36:11.667492 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed0485-bc1c-489c-93f6-cd88325f3f29 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:11.669026 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-cbed0485-bc1c-489c-93f6-cd88325f3f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:11.692533 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbed0485-bc1c-489c-93f6-cd88325f3f29 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:11.763627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a88dc23-dfe9-4c09-9846-10a0b533cb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:11.766563 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88dc23-dfe9-4c09-9846-10a0b533cb01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:11.766563 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88dc23-dfe9-4c09-9846-10a0b533cb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:11.766563 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a88dc23-dfe9-4c09-9846-10a0b533cb01 admin admin] PATH_INFO: `/v3/projects/8e55ff60340f48558375780f4887a9a9` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:11.767966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a88dc23-dfe9-4c09-9846-10a0b533cb01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e55ff60340f48558375780f4887a9a9)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:11.772638 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a88dc23-dfe9-4c09-9846-10a0b533cb01 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:11.837337 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2a88dc23-dfe9-4c09-9846-10a0b533cb01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e55ff60340f48558375780f4887a9a9', 'request_id': 'req-2a88dc23-dfe9-4c09-9846-10a0b533cb01'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:11.853312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2a88dc23-dfe9-4c09-9846-10a0b533cb01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e55ff60340f48558375780f4887a9a9', 'request_id': 'req-2a88dc23-dfe9-4c09-9846-10a0b533cb01'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:11.863480 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2a88dc23-dfe9-4c09-9846-10a0b533cb01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e55ff60340f48558375780f4887a9a9', 'request_id': 'req-2a88dc23-dfe9-4c09-9846-10a0b533cb01'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:11.881531 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2a88dc23-dfe9-4c09-9846-10a0b533cb01 admin admin] The token cache is being invalidate because project 8e55ff60340f48558375780f4887a9a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:36:11.881872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2a88dc23-dfe9-4c09-9846-10a0b533cb01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:11.882670 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2a88dc23-dfe9-4c09-9846-10a0b533cb01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:11.884722 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1329/2674] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:36:11 2022] DELETE /identity/v3/projects/8e55ff60340f48558375780f4887a9a9 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:36:11.890406 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b37ea-e360-4a78-9265-43955260cfa7 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:11.890804 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b37ea-e360-4a78-9265-43955260cfa7 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:11.891171 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b37ea-e360-4a78-9265-43955260cfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:11.919224 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f25b37ea-e360-4a78-9265-43955260cfa7 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:36:11.920649 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f25b37ea-e360-4a78-9265-43955260cfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:11.945307 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:12.017898 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1346/2675] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:36:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:36:12.019783 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f7802-3972-4080-bd7c-6eca343df106 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:12.025432 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f7802-3972-4080-bd7c-6eca343df106 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:12.026242 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f7802-3972-4080-bd7c-6eca343df106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:12.054054 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d27f7802-3972-4080-bd7c-6eca343df106 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:36:12.055460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d27f7802-3972-4080-bd7c-6eca343df106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:12.067668 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1330/2676] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:36:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:36:12.072094 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b37ea-e360-4a78-9265-43955260cfa7 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:12.084112 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b37ea-e360-4a78-9265-43955260cfa7 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:12.089385 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39df07-d1be-4926-91da-9bc0c810f6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:12.090016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39df07-d1be-4926-91da-9bc0c810f6ef admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:12.090016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39df07-d1be-4926-91da-9bc0c810f6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:12.091352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c39df07-d1be-4926-91da-9bc0c810f6ef admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:12.092279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c39df07-d1be-4926-91da-9bc0c810f6ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:36:12.092591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c39df07-d1be-4926-91da-9bc0c810f6ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:12.094260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c39df07-d1be-4926-91da-9bc0c810f6ef admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:12.109481 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1347/2677] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:36:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:36:12.114067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f7802-3972-4080-bd7c-6eca343df106 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:12.125582 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f7802-3972-4080-bd7c-6eca343df106 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:12.131491 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f92076-0946-480d-b3c7-e1737a03e939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:12.132106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f92076-0946-480d-b3c7-e1737a03e939 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:12.132381 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f92076-0946-480d-b3c7-e1737a03e939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:12.132856 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f92076-0946-480d-b3c7-e1737a03e939 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:12.133494 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f92076-0946-480d-b3c7-e1737a03e939 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:12.135633 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f92076-0946-480d-b3c7-e1737a03e939 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:12.179286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1331/2678] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:36:12 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:36:12.180401 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39df07-d1be-4926-91da-9bc0c810f6ef admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:12.194864 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c39df07-d1be-4926-91da-9bc0c810f6ef admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:12.278791 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fad65e-62ec-4646-b327-a1f4fba990ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:12.279547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fad65e-62ec-4646-b327-a1f4fba990ed admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:12.280157 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fad65e-62ec-4646-b327-a1f4fba990ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:12.280548 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fad65e-62ec-4646-b327-a1f4fba990ed admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:12.281294 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fad65e-62ec-4646-b327-a1f4fba990ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:12.283356 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fad65e-62ec-4646-b327-a1f4fba990ed admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:12.317920 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1348/2679] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:36:12 2022] POST /identity/v3/users => generated 292 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:36:12.322253 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f92076-0946-480d-b3c7-e1737a03e939 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:12.335516 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f92076-0946-480d-b3c7-e1737a03e939 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:12.341821 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f07d3-ae71-4543-b1ec-e3c19c654b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:12.342439 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f07d3-ae71-4543-b1ec-e3c19c654b52 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:12.342720 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f07d3-ae71-4543-b1ec-e3c19c654b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:12.342992 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f07d3-ae71-4543-b1ec-e3c19c654b52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:12.343750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f07d3-ae71-4543-b1ec-e3c19c654b52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:12.345605 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f07d3-ae71-4543-b1ec-e3c19c654b52 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:12.369050 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1332/2680] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:36:12.373523 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fad65e-62ec-4646-b327-a1f4fba990ed admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:12.385717 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fad65e-62ec-4646-b327-a1f4fba990ed admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:12.390839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f3ab07-85bd-49eb-94a2-64b68f6add7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:12.391739 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3ab07-85bd-49eb-94a2-64b68f6add7b admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:12.392027 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3ab07-85bd-49eb-94a2-64b68f6add7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:12.393016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f3ab07-85bd-49eb-94a2-64b68f6add7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:12.393016 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3ab07-85bd-49eb-94a2-64b68f6add7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:12.394652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f3ab07-85bd-49eb-94a2-64b68f6add7b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:12.409694 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1349/2681] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:36:12.412348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f07d3-ae71-4543-b1ec-e3c19c654b52 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:12.423515 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f07d3-ae71-4543-b1ec-e3c19c654b52 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:12.428584 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06413fa5-e350-4fd9-bc91-df8ca3e7abbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:12.429435 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06413fa5-e350-4fd9-bc91-df8ca3e7abbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:12.434395 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06413fa5-e350-4fd9-bc91-df8ca3e7abbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:12.434395 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06413fa5-e350-4fd9-bc91-df8ca3e7abbd admin admin] PATH_INFO: `/v3/projects/cbe49cbe2ef040d5b7422f551dfc30ca/users/dc0975c6febb4a83ae59b623d064c7c7/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:12.443611 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:12.465517 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06413fa5-e350-4fd9-bc91-df8ca3e7abbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbe49cbe2ef040d5b7422f551dfc30ca, user_id=dc0975c6febb4a83ae59b623d064c7c7, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:12.468460 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06413fa5-e350-4fd9-bc91-df8ca3e7abbd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:12.487891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1333/2682] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:36:12 2022] PUT /identity/v3/projects/cbe49cbe2ef040d5b7422f551dfc30ca/users/dc0975c6febb4a83ae59b623d064c7c7/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:36:12.502681 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a21d8-ef88-47da-889b-eb19a7661492 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:12.502681 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a21d8-ef88-47da-889b-eb19a7661492 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:12.502958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07a21d8-ef88-47da-889b-eb19a7661492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:12.558362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d07a21d8-ef88-47da-889b-eb19a7661492 None None] MFA Rules not processed for user `dc0975c6febb4a83ae59b623d064c7c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:36:12.558362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d07a21d8-ef88-47da-889b-eb19a7661492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:12.635061 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1350/2683] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:36:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:36:12.639958 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06413fa5-e350-4fd9-bc91-df8ca3e7abbd admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:12.733397 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06413fa5-e350-4fd9-bc91-df8ca3e7abbd admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:12.738179 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00269ba-6442-4e08-8688-102f04c0dc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:12.738812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00269ba-6442-4e08-8688-102f04c0dc43 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:12.739086 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00269ba-6442-4e08-8688-102f04c0dc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:12.739351 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00269ba-6442-4e08-8688-102f04c0dc43 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:12.740001 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00269ba-6442-4e08-8688-102f04c0dc43 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:12.741890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00269ba-6442-4e08-8688-102f04c0dc43 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:12.781975 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1334/2684] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:36:12 2022] POST /identity/v3/projects => generated 383 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:36:12.787191 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a21d8-ef88-47da-889b-eb19a7661492 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:12.804687 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07a21d8-ef88-47da-889b-eb19a7661492 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:12.820273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:12.889762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac53660-dce5-4692-af37-caa34376c317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:12.890314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac53660-dce5-4692-af37-caa34376c317 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:12.890592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac53660-dce5-4692-af37-caa34376c317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:12.890861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac53660-dce5-4692-af37-caa34376c317 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:12.891483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac53660-dce5-4692-af37-caa34376c317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:12.893430 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac53660-dce5-4692-af37-caa34376c317 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:12.924396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1351/2685] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:36:12 2022] POST /identity/v3/users => generated 298 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:36:12.928145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00269ba-6442-4e08-8688-102f04c0dc43 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:12.940072 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00269ba-6442-4e08-8688-102f04c0dc43 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:12.945176 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5e2a9-0fc3-45c0-bc01-02c7fefeca8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:12.945869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5e2a9-0fc3-45c0-bc01-02c7fefeca8d admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:12.946260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5e2a9-0fc3-45c0-bc01-02c7fefeca8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:12.946619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b5e2a9-0fc3-45c0-bc01-02c7fefeca8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:12.947334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5e2a9-0fc3-45c0-bc01-02c7fefeca8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:12.948996 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b5e2a9-0fc3-45c0-bc01-02c7fefeca8d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:12.964420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1335/2686] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:36:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:36:12.966160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac53660-dce5-4692-af37-caa34376c317 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:12.977003 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac53660-dce5-4692-af37-caa34376c317 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:12.982844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694412a-9b93-4bf4-b427-e44a7276ac6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:12.983527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694412a-9b93-4bf4-b427-e44a7276ac6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:12.983809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694412a-9b93-4bf4-b427-e44a7276ac6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:12.984086 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6694412a-9b93-4bf4-b427-e44a7276ac6a admin admin] PATH_INFO: `/v3/projects/59a0646832334456b590bca2590b872a/users/5df2b001fc44473b8ba9142520d2b6d8/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:13.003764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694412a-9b93-4bf4-b427-e44a7276ac6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a0646832334456b590bca2590b872a, user_id=5df2b001fc44473b8ba9142520d2b6d8, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:13.006570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6694412a-9b93-4bf4-b427-e44a7276ac6a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:13.026494 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1352/2687] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:36:12 2022] PUT /identity/v3/projects/59a0646832334456b590bca2590b872a/users/5df2b001fc44473b8ba9142520d2b6d8/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:36:13.031172 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5e2a9-0fc3-45c0-bc01-02c7fefeca8d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:13.041943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b5e2a9-0fc3-45c0-bc01-02c7fefeca8d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:13.131432 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e08af75-31ec-4126-aee8-76eece0bb568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:13.132064 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08af75-31ec-4126-aee8-76eece0bb568 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:13.132341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08af75-31ec-4126-aee8-76eece0bb568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:13.132605 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e08af75-31ec-4126-aee8-76eece0bb568 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:13.133239 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08af75-31ec-4126-aee8-76eece0bb568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:13.134896 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e08af75-31ec-4126-aee8-76eece0bb568 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:13.160248 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1336/2688] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:36:13 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:36:13.164416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694412a-9b93-4bf4-b427-e44a7276ac6a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:13.175720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6694412a-9b93-4bf4-b427-e44a7276ac6a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:13.180708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b7c1f-0477-4e0c-ad18-8f424ea3c5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:13.181489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b7c1f-0477-4e0c-ad18-8f424ea3c5c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:13.181839 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b7c1f-0477-4e0c-ad18-8f424ea3c5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:13.182243 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649b7c1f-0477-4e0c-ad18-8f424ea3c5c2 admin admin] PATH_INFO: `/v3/projects/59a0646832334456b590bca2590b872a/users/5df2b001fc44473b8ba9142520d2b6d8/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:13.184312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b7c1f-0477-4e0c-ad18-8f424ea3c5c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a0646832334456b590bca2590b872a, user_id=5df2b001fc44473b8ba9142520d2b6d8, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:13.188180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649b7c1f-0477-4e0c-ad18-8f424ea3c5c2 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:13.205372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1353/2689] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:36:13 2022] PUT /identity/v3/projects/59a0646832334456b590bca2590b872a/users/5df2b001fc44473b8ba9142520d2b6d8/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:36:13.215589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee815de-af76-4823-9716-e7dc4c383887 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:13.216292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee815de-af76-4823-9716-e7dc4c383887 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:13.216784 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee815de-af76-4823-9716-e7dc4c383887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:13.252950 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-5ee815de-af76-4823-9716-e7dc4c383887 None None] MFA Rules not processed for user `5df2b001fc44473b8ba9142520d2b6d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:36:13.254289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5ee815de-af76-4823-9716-e7dc4c383887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:13.315181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:13.352795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1337/2690] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:36:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:36:13.371493 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b7c1f-0477-4e0c-ad18-8f424ea3c5c2 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:13.373658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-649b7c1f-0477-4e0c-ad18-8f424ea3c5c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:13.463377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649b7c1f-0477-4e0c-ad18-8f424ea3c5c2 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:13.469762 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e00ca-06bd-4950-8a65-f3c849f82a52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:13.470567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e00ca-06bd-4950-8a65-f3c849f82a52 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:13.470950 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e00ca-06bd-4950-8a65-f3c849f82a52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:13.471324 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2e00ca-06bd-4950-8a65-f3c849f82a52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:13.557562 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e00ca-06bd-4950-8a65-f3c849f82a52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:13.561466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2e00ca-06bd-4950-8a65-f3c849f82a52 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:13.586626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1354/2691] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:36:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:36:18.328552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee815de-af76-4823-9716-e7dc4c383887 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:18.330000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-5ee815de-af76-4823-9716-e7dc4c383887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:18.341750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee815de-af76-4823-9716-e7dc4c383887 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:18.345288 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c86a1d-32d0-4c62-bffe-bfa73e9e9b68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:18.345877 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c86a1d-32d0-4c62-bffe-bfa73e9e9b68 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:18.346152 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c86a1d-32d0-4c62-bffe-bfa73e9e9b68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:18.346446 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c86a1d-32d0-4c62-bffe-bfa73e9e9b68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:18.355854 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c86a1d-32d0-4c62-bffe-bfa73e9e9b68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:18.357262 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c86a1d-32d0-4c62-bffe-bfa73e9e9b68 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:18.366922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1338/2692] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:36:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:36:23.433237 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e00ca-06bd-4950-8a65-f3c849f82a52 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:23.463398 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:23.503799 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2e00ca-06bd-4950-8a65-f3c849f82a52 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:23.509164 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6e700-2e76-4688-a725-22c7ce2f857b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:23.509810 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6e700-2e76-4688-a725-22c7ce2f857b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:23.510093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6e700-2e76-4688-a725-22c7ce2f857b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:23.510399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd6e700-2e76-4688-a725-22c7ce2f857b admin admin] PATH_INFO: `/v3/users/dc0975c6febb4a83ae59b623d064c7c7` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:23.511581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6e700-2e76-4688-a725-22c7ce2f857b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc0975c6febb4a83ae59b623d064c7c7)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:23.513537 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd6e700-2e76-4688-a725-22c7ce2f857b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:23.565288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bdd6e700-2e76-4688-a725-22c7ce2f857b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc0975c6febb4a83ae59b623d064c7c7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:23.573725 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bdd6e700-2e76-4688-a725-22c7ce2f857b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc0975c6febb4a83ae59b623d064c7c7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:23.587976 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bdd6e700-2e76-4688-a725-22c7ce2f857b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc0975c6febb4a83ae59b623d064c7c7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:23.588298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bdd6e700-2e76-4688-a725-22c7ce2f857b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc0975c6febb4a83ae59b623d064c7c7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:23.596669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-bdd6e700-2e76-4688-a725-22c7ce2f857b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc0975c6febb4a83ae59b623d064c7c7'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:23.604722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1355/2693] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:36:23 2022] DELETE /identity/v3/users/dc0975c6febb4a83ae59b623d064c7c7 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:36:23.610479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82ff01-8799-4bab-9911-8803a7ba87ec None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:23.611047 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82ff01-8799-4bab-9911-8803a7ba87ec None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:23.611204 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c82ff01-8799-4bab-9911-8803a7ba87ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:23.630893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4c82ff01-8799-4bab-9911-8803a7ba87ec None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:36:23.632183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4c82ff01-8799-4bab-9911-8803a7ba87ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:23.694493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1339/2694] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:36:23.704741 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6e700-2e76-4688-a725-22c7ce2f857b admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:23.706257 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bdd6e700-2e76-4688-a725-22c7ce2f857b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:23.766070 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd6e700-2e76-4688-a725-22c7ce2f857b admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:23.769701 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d480f5-c478-4f40-a4c2-87109bc8a406 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:23.770344 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d480f5-c478-4f40-a4c2-87109bc8a406 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:23.770620 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d480f5-c478-4f40-a4c2-87109bc8a406 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:23.770889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d480f5-c478-4f40-a4c2-87109bc8a406 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:23.780342 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d480f5-c478-4f40-a4c2-87109bc8a406 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:23.781790 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d480f5-c478-4f40-a4c2-87109bc8a406 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:23.792013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1356/2695] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:36:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:36:24.074354 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82ff01-8799-4bab-9911-8803a7ba87ec None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:24.079672 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4c82ff01-8799-4bab-9911-8803a7ba87ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:24.099882 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c82ff01-8799-4bab-9911-8803a7ba87ec None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:24.106193 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:24.106805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:24.107081 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:24.107348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a admin admin] PATH_INFO: `/v3/projects/cbe49cbe2ef040d5b7422f551dfc30ca` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:24.108478 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbe49cbe2ef040d5b7422f551dfc30ca)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:24.110405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:24.161557 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbe49cbe2ef040d5b7422f551dfc30ca', 'request_id': 'req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:24.167083 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbe49cbe2ef040d5b7422f551dfc30ca', 'request_id': 'req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:24.176670 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbe49cbe2ef040d5b7422f551dfc30ca', 'request_id': 'req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:24.189776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a admin admin] The token cache is being invalidate because project cbe49cbe2ef040d5b7422f551dfc30ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:36:24.190177 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:24.190986 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:24.192491 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1340/2696] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:36:24 2022] DELETE /identity/v3/projects/cbe49cbe2ef040d5b7422f551dfc30ca => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:36:24.196729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d480f5-c478-4f40-a4c2-87109bc8a406 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:24.206859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d480f5-c478-4f40-a4c2-87109bc8a406 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:24.254635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:24.271287 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879b523-f178-480a-9e1a-628402b0b81f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:24.271981 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879b523-f178-480a-9e1a-628402b0b81f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:24.272787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879b523-f178-480a-9e1a-628402b0b81f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:24.272787 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879b523-f178-480a-9e1a-628402b0b81f admin admin] PATH_INFO: `/v3/users/5df2b001fc44473b8ba9142520d2b6d8` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:24.273682 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879b523-f178-480a-9e1a-628402b0b81f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5df2b001fc44473b8ba9142520d2b6d8)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:24.276180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879b523-f178-480a-9e1a-628402b0b81f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:24.329900 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2879b523-f178-480a-9e1a-628402b0b81f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5df2b001fc44473b8ba9142520d2b6d8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:24.341355 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2879b523-f178-480a-9e1a-628402b0b81f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5df2b001fc44473b8ba9142520d2b6d8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:24.358874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2879b523-f178-480a-9e1a-628402b0b81f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5df2b001fc44473b8ba9142520d2b6d8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:24.360133 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2879b523-f178-480a-9e1a-628402b0b81f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5df2b001fc44473b8ba9142520d2b6d8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:24.368670 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2879b523-f178-480a-9e1a-628402b0b81f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5df2b001fc44473b8ba9142520d2b6d8'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:24.375273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1357/2697] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:36:24 2022] DELETE /identity/v3/users/5df2b001fc44473b8ba9142520d2b6d8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:36:24.687323 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:24.689546 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:24.717998 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b90f4c-17cd-42a7-a4fd-d32e7d4d891a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:24.765928 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:24.780612 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59699df5-97ad-4e7a-b524-fb736a2e4c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:24.781354 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59699df5-97ad-4e7a-b524-fb736a2e4c94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:24.781637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59699df5-97ad-4e7a-b524-fb736a2e4c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:24.781913 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59699df5-97ad-4e7a-b524-fb736a2e4c94 admin admin] PATH_INFO: `/v3/projects/59a0646832334456b590bca2590b872a` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:24.783128 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59699df5-97ad-4e7a-b524-fb736a2e4c94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a0646832334456b590bca2590b872a)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:24.785604 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59699df5-97ad-4e7a-b524-fb736a2e4c94 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:24.834861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-59699df5-97ad-4e7a-b524-fb736a2e4c94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a0646832334456b590bca2590b872a', 'request_id': 'req-59699df5-97ad-4e7a-b524-fb736a2e4c94'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:24.840411 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-59699df5-97ad-4e7a-b524-fb736a2e4c94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a0646832334456b590bca2590b872a', 'request_id': 'req-59699df5-97ad-4e7a-b524-fb736a2e4c94'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:24.847978 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-59699df5-97ad-4e7a-b524-fb736a2e4c94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a0646832334456b590bca2590b872a', 'request_id': 'req-59699df5-97ad-4e7a-b524-fb736a2e4c94'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:24.859461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-59699df5-97ad-4e7a-b524-fb736a2e4c94 admin admin] The token cache is being invalidate because project 59a0646832334456b590bca2590b872a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:36:24.859744 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-59699df5-97ad-4e7a-b524-fb736a2e4c94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:24.860412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-59699df5-97ad-4e7a-b524-fb736a2e4c94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:24.861876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1341/2698] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:36:24 2022] DELETE /identity/v3/projects/59a0646832334456b590bca2590b872a => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:36:24.867895 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648a9ff-8f1c-4a92-b6bf-bed25f7466c4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:24.868401 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648a9ff-8f1c-4a92-b6bf-bed25f7466c4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:24.868507 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5648a9ff-8f1c-4a92-b6bf-bed25f7466c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:24.889552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5648a9ff-8f1c-4a92-b6bf-bed25f7466c4 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:36:24.890801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5648a9ff-8f1c-4a92-b6bf-bed25f7466c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:24.943437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be0b5b-7a15-4729-bdb9-4f588435aed6 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:24.943726 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be0b5b-7a15-4729-bdb9-4f588435aed6 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:24.943998 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7be0b5b-7a15-4729-bdb9-4f588435aed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:24.957335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1358/2699] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:36:24.964401 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e7be0b5b-7a15-4729-bdb9-4f588435aed6 None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:36:24.965534 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e7be0b5b-7a15-4729-bdb9-4f588435aed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:24.965669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51dc63-b0ab-4013-802f-80464b2b1ae4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:24.965934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51dc63-b0ab-4013-802f-80464b2b1ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:24.966227 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b51dc63-b0ab-4013-802f-80464b2b1ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:24.985603 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8b51dc63-b0ab-4013-802f-80464b2b1ae4 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:36:24.986861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8b51dc63-b0ab-4013-802f-80464b2b1ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:24.995513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1359/2700] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:36:25.000931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51dc63-b0ab-4013-802f-80464b2b1ae4 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:25.013857 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b51dc63-b0ab-4013-802f-80464b2b1ae4 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:25.020834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5978e-e60d-4b9d-8cd5-a3b180a94b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:25.021446 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5978e-e60d-4b9d-8cd5-a3b180a94b43 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:25.021746 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5978e-e60d-4b9d-8cd5-a3b180a94b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:25.021988 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5978e-e60d-4b9d-8cd5-a3b180a94b43 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:25.022664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5978e-e60d-4b9d-8cd5-a3b180a94b43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:36:25.023006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5978e-e60d-4b9d-8cd5-a3b180a94b43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:25.024544 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce5978e-e60d-4b9d-8cd5-a3b180a94b43 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:25.031071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1342/2701] 10.222.0.76 () {58 vars in 1009 bytes} [Wed Aug 24 15:36:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:36:25.041485 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1360/2702] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 15:36:25.044666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be0b5b-7a15-4729-bdb9-4f588435aed6 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:25.046643 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5978e-e60d-4b9d-8cd5-a3b180a94b43 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:25.046940 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e7be0b5b-7a15-4729-bdb9-4f588435aed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:25.059707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5978e-e60d-4b9d-8cd5-a3b180a94b43 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:25.064569 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee37a29-f1a5-4a48-bfef-1945d16ae460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:25.065139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee37a29-f1a5-4a48-bfef-1945d16ae460 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:25.065417 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee37a29-f1a5-4a48-bfef-1945d16ae460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:25.065807 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee37a29-f1a5-4a48-bfef-1945d16ae460 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:25.066336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee37a29-f1a5-4a48-bfef-1945d16ae460 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:25.068239 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee37a29-f1a5-4a48-bfef-1945d16ae460 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:25.111058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1361/2703] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:36:25 2022] POST /identity/v3/projects => generated 397 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:36:25.112106 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee37a29-f1a5-4a48-bfef-1945d16ae460 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:25.127287 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7be0b5b-7a15-4729-bdb9-4f588435aed6 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:25.130749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:25.153662 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:25.184679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee37a29-f1a5-4a48-bfef-1945d16ae460 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:25.189638 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe34add-fd6c-4a45-ae11-447fe9226c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:25.190361 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe34add-fd6c-4a45-ae11-447fe9226c44 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:25.190792 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe34add-fd6c-4a45-ae11-447fe9226c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:25.191029 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe34add-fd6c-4a45-ae11-447fe9226c44 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:25.191664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe34add-fd6c-4a45-ae11-447fe9226c44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:25.193615 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe34add-fd6c-4a45-ae11-447fe9226c44 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:25.201582 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160276a5-314b-4832-8f1b-ba344d92c1cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:25.202740 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160276a5-314b-4832-8f1b-ba344d92c1cf service swift] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:25.203676 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160276a5-314b-4832-8f1b-ba344d92c1cf service swift] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:25.204704 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160276a5-314b-4832-8f1b-ba344d92c1cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:25.217334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1362/2704] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:36:25 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:36:25.223007 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe34add-fd6c-4a45-ae11-447fe9226c44 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:25.234888 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe34add-fd6c-4a45-ae11-447fe9226c44 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:25.240186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0481ab-7a8f-422d-810f-c015aa10da74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:25.240903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0481ab-7a8f-422d-810f-c015aa10da74 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:25.241248 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0481ab-7a8f-422d-810f-c015aa10da74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:25.241581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0481ab-7a8f-422d-810f-c015aa10da74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:25.242255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0481ab-7a8f-422d-810f-c015aa10da74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:25.243934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0481ab-7a8f-422d-810f-c015aa10da74 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:25.256963 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1363/2705] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:36:25.261051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0481ab-7a8f-422d-810f-c015aa10da74 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:25.271394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0481ab-7a8f-422d-810f-c015aa10da74 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:25.276352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6845b634-7dca-4457-b781-1048ae7c575e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:25.276582 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160276a5-314b-4832-8f1b-ba344d92c1cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:25.276999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845b634-7dca-4457-b781-1048ae7c575e admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:25.277277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845b634-7dca-4457-b781-1048ae7c575e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:25.277547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6845b634-7dca-4457-b781-1048ae7c575e admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:25.278261 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845b634-7dca-4457-b781-1048ae7c575e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:25.278393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160276a5-314b-4832-8f1b-ba344d92c1cf service swift] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:25.279983 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6845b634-7dca-4457-b781-1048ae7c575e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:25.287479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1343/2706] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:36:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:36:25.295372 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1364/2707] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 15:36:25.301211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160276a5-314b-4832-8f1b-ba344d92c1cf service swift] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:25.317480 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160276a5-314b-4832-8f1b-ba344d92c1cf service swift] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:25.322341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4eedc7-8ec6-4635-83eb-edce1c52fa35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:25.323095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4eedc7-8ec6-4635-83eb-edce1c52fa35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:25.323368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4eedc7-8ec6-4635-83eb-edce1c52fa35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:25.323687 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4eedc7-8ec6-4635-83eb-edce1c52fa35 admin admin] PATH_INFO: `/v3/projects/0fc1e123699845dd97b447eb2b65853e/users/5bd9db8d627641c0acb8bfcf298aa113/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:25.346057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4eedc7-8ec6-4635-83eb-edce1c52fa35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fc1e123699845dd97b447eb2b65853e, user_id=5bd9db8d627641c0acb8bfcf298aa113, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:25.348747 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4eedc7-8ec6-4635-83eb-edce1c52fa35 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:25.366272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1344/2708] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:36:25 2022] PUT /identity/v3/projects/0fc1e123699845dd97b447eb2b65853e/users/5bd9db8d627641c0acb8bfcf298aa113/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:36:25.381759 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb954767-815d-4ad9-a425-354ab3d86ef2 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:25.382345 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb954767-815d-4ad9-a425-354ab3d86ef2 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:25.383238 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb954767-815d-4ad9-a425-354ab3d86ef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:25.431156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-fb954767-815d-4ad9-a425-354ab3d86ef2 None None] MFA Rules not processed for user `5bd9db8d627641c0acb8bfcf298aa113`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:36:25.434519 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fb954767-815d-4ad9-a425-354ab3d86ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:25.581707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1365/2709] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:36:25 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:36:25.597591 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4eedc7-8ec6-4635-83eb-edce1c52fa35 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:25.602164 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-af4eedc7-8ec6-4635-83eb-edce1c52fa35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:25.678101 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb954767-815d-4ad9-a425-354ab3d86ef2 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:25.679707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fb954767-815d-4ad9-a425-354ab3d86ef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:25.695393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4eedc7-8ec6-4635-83eb-edce1c52fa35 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:25.699478 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a07659-b6ce-4b72-a28a-2485e1da3928 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:25.700221 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a07659-b6ce-4b72-a28a-2485e1da3928 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:25.700598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a07659-b6ce-4b72-a28a-2485e1da3928 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:25.701012 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a07659-b6ce-4b72-a28a-2485e1da3928 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:25.711263 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a07659-b6ce-4b72-a28a-2485e1da3928 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:25.718500 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a07659-b6ce-4b72-a28a-2485e1da3928 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:25.729537 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1345/2710] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:36:25 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:36:25.782318 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb954767-815d-4ad9-a425-354ab3d86ef2 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:25.787352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f021bbe-09ad-4fa2-a3f1-ec6c2363bd2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:25.787979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f021bbe-09ad-4fa2-a3f1-ec6c2363bd2f service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:25.788456 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f021bbe-09ad-4fa2-a3f1-ec6c2363bd2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:25.788833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f021bbe-09ad-4fa2-a3f1-ec6c2363bd2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:25.790738 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2f021bbe-09ad-4fa2-a3f1-ec6c2363bd2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:25.831546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:25.895823 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f021bbe-09ad-4fa2-a3f1-ec6c2363bd2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:25.897804 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f021bbe-09ad-4fa2-a3f1-ec6c2363bd2f service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:25.912993 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1366/2711] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:36:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:36:27.165546 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a07659-b6ce-4b72-a28a-2485e1da3928 service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:27.167586 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b3a07659-b6ce-4b72-a28a-2485e1da3928 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:27.180836 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a07659-b6ce-4b72-a28a-2485e1da3928 service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:27.185213 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c6861-438c-4ed6-a680-4c5b6f42fc40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:27.185830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c6861-438c-4ed6-a680-4c5b6f42fc40 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:27.186108 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c6861-438c-4ed6-a680-4c5b6f42fc40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:27.186413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5c6861-438c-4ed6-a680-4c5b6f42fc40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:27.188154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-db5c6861-438c-4ed6-a680-4c5b6f42fc40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:27.202914 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c6861-438c-4ed6-a680-4c5b6f42fc40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:27.204345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5c6861-438c-4ed6-a680-4c5b6f42fc40 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:27.216224 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1346/2712] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:36:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:36:33.302022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f021bbe-09ad-4fa2-a3f1-ec6c2363bd2f service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:33.303930 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2f021bbe-09ad-4fa2-a3f1-ec6c2363bd2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:33.318354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f021bbe-09ad-4fa2-a3f1-ec6c2363bd2f service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:33.324142 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3c60c7-9610-4968-b8fc-24d0281e8f6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:33.324755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c60c7-9610-4968-b8fc-24d0281e8f6e service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:33.325035 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c60c7-9610-4968-b8fc-24d0281e8f6e service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:33.325841 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3c60c7-9610-4968-b8fc-24d0281e8f6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:33.327667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2d3c60c7-9610-4968-b8fc-24d0281e8f6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:33.412037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c60c7-9610-4968-b8fc-24d0281e8f6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:33.413628 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3c60c7-9610-4968-b8fc-24d0281e8f6e service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:33.423404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1367/2713] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:36:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:36:37.210864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:38.303361 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c6861-438c-4ed6-a680-4c5b6f42fc40 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:38.372471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5c6861-438c-4ed6-a680-4c5b6f42fc40 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:38.377026 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37937b-5a49-4a1a-8d2b-30080cdee999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:38.377656 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37937b-5a49-4a1a-8d2b-30080cdee999 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:38.377929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37937b-5a49-4a1a-8d2b-30080cdee999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:38.378194 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a37937b-5a49-4a1a-8d2b-30080cdee999 admin admin] PATH_INFO: `/v3/users/5bd9db8d627641c0acb8bfcf298aa113` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:38.379335 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37937b-5a49-4a1a-8d2b-30080cdee999 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bd9db8d627641c0acb8bfcf298aa113)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:38.381235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a37937b-5a49-4a1a-8d2b-30080cdee999 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:38.428196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2a37937b-5a49-4a1a-8d2b-30080cdee999 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bd9db8d627641c0acb8bfcf298aa113'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:38.428861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2a37937b-5a49-4a1a-8d2b-30080cdee999 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bd9db8d627641c0acb8bfcf298aa113'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:38.441154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2a37937b-5a49-4a1a-8d2b-30080cdee999 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bd9db8d627641c0acb8bfcf298aa113'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:38.449176 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2a37937b-5a49-4a1a-8d2b-30080cdee999 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bd9db8d627641c0acb8bfcf298aa113'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:38.449837 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-2a37937b-5a49-4a1a-8d2b-30080cdee999 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bd9db8d627641c0acb8bfcf298aa113'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:38.463704 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1347/2714] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:36:38 2022] DELETE /identity/v3/users/5bd9db8d627641c0acb8bfcf298aa113 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:36:38.468210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948382c-bee2-4827-a64b-ed6e9db71824 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:38.468957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948382c-bee2-4827-a64b-ed6e9db71824 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:38.468957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5948382c-bee2-4827-a64b-ed6e9db71824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:38.487679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-5948382c-bee2-4827-a64b-ed6e9db71824 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:36:38.488906 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5948382c-bee2-4827-a64b-ed6e9db71824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:38.549667 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1368/2715] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:36:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:36:38.559399 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37937b-5a49-4a1a-8d2b-30080cdee999 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:38.561338 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2a37937b-5a49-4a1a-8d2b-30080cdee999 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:38.618748 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a37937b-5a49-4a1a-8d2b-30080cdee999 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:38.622895 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7262f49-48d5-48ca-af8e-741b8faf07a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:38.623497 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7262f49-48d5-48ca-af8e-741b8faf07a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:38.623767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7262f49-48d5-48ca-af8e-741b8faf07a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:38.624030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7262f49-48d5-48ca-af8e-741b8faf07a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:38.633737 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7262f49-48d5-48ca-af8e-741b8faf07a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:38.635219 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7262f49-48d5-48ca-af8e-741b8faf07a3 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:38.645234 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1348/2716] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:36:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:36:38.932435 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948382c-bee2-4827-a64b-ed6e9db71824 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:38.933880 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-5948382c-bee2-4827-a64b-ed6e9db71824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:38.946830 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5948382c-bee2-4827-a64b-ed6e9db71824 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:38.951688 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:38.952364 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:38.952645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:38.952923 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe admin admin] PATH_INFO: `/v3/projects/0fc1e123699845dd97b447eb2b65853e` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:38.954076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fc1e123699845dd97b447eb2b65853e)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:38.956959 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:38.992642 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:39.011594 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fc1e123699845dd97b447eb2b65853e', 'request_id': 'req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:39.018707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fc1e123699845dd97b447eb2b65853e', 'request_id': 'req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:39.030957 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fc1e123699845dd97b447eb2b65853e', 'request_id': 'req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:39.043376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe admin admin] The token cache is being invalidate because project 0fc1e123699845dd97b447eb2b65853e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:36:39.044574 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:39.045266 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:36:39.046867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1369/2717] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:36:38 2022] DELETE /identity/v3/projects/0fc1e123699845dd97b447eb2b65853e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:36:45.675034 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacc2cb-2be4-4470-98e5-5f9c96311dab None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:45.675474 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacc2cb-2be4-4470-98e5-5f9c96311dab None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:45.675601 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacc2cb-2be4-4470-98e5-5f9c96311dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:45.706293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:45.725917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-4aacc2cb-2be4-4470-98e5-5f9c96311dab None None] MFA Rules not processed for user `ad682eade4104a799b21f47aa7cdd4f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:36:45.727189 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4aacc2cb-2be4-4470-98e5-5f9c96311dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:45.789554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1349/2718] 10.222.0.76 () {58 vars in 967 bytes} [Wed Aug 24 15:36:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:36:45.801512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:36:45.802969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:36:45.892170 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f1e1ab-9ec1-4c3a-8370-b3230e3ecebe admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:36:45.897219 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b3c7d-b1e6-4ffb-9f34-4b12fc4e072d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0679072b37db41e880a95b738bea47ea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:36:45.897816 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b3c7d-b1e6-4ffb-9f34-4b12fc4e072d service swift] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:36:45.898092 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b3c7d-b1e6-4ffb-9f34-4b12fc4e072d service swift] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:36:45.898390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381b3c7d-b1e6-4ffb-9f34-4b12fc4e072d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:36:45.906568 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b3c7d-b1e6-4ffb-9f34-4b12fc4e072d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:36:45.908163 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381b3c7d-b1e6-4ffb-9f34-4b12fc4e072d service swift] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:36:45.916579 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1370/2719] 10.222.0.76 () {60 vars in 1384 bytes} [Wed Aug 24 15:36:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 15:36:55.786494 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=30 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:36:55.913115 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=55,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:48.961862 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aacc2cb-2be4-4470-98e5-5f9c96311dab None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:48.963866 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-4aacc2cb-2be4-4470-98e5-5f9c96311dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:49.022131 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aacc2cb-2be4-4470-98e5-5f9c96311dab None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:49.025644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43462894-912f-4c76-b1f5-25116d3a27b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:49.026306 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43462894-912f-4c76-b1f5-25116d3a27b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:49.026593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43462894-912f-4c76-b1f5-25116d3a27b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:49.026861 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43462894-912f-4c76-b1f5-25116d3a27b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:49.029076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-43462894-912f-4c76-b1f5-25116d3a27b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:49.089992 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43462894-912f-4c76-b1f5-25116d3a27b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:49.092127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43462894-912f-4c76-b1f5-25116d3a27b6 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:49.101791 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1350/2720] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:52.410097 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b3c7d-b1e6-4ffb-9f34-4b12fc4e072d service swift] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:52.411915 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-381b3c7d-b1e6-4ffb-9f34-4b12fc4e072d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:52.474698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43462894-912f-4c76-b1f5-25116d3a27b6 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:52.476645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-43462894-912f-4c76-b1f5-25116d3a27b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:52.489183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381b3c7d-b1e6-4ffb-9f34-4b12fc4e072d service swift] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:52.489932 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43462894-912f-4c76-b1f5-25116d3a27b6 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:52.494589 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb99f9a-22da-47f4-aae4-158d8a354727 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:52.494769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98703153-e0ff-44a9-a88f-331c49fd3ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:52.495211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98703153-e0ff-44a9-a88f-331c49fd3ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:52.495404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb99f9a-22da-47f4-aae4-158d8a354727 service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:52.495526 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98703153-e0ff-44a9-a88f-331c49fd3ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:52.495746 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98703153-e0ff-44a9-a88f-331c49fd3ffd admin admin] PATH_INFO: `/v3/users/56535a702f4344aa8a58f11406acf4c9` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:52.495853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb99f9a-22da-47f4-aae4-158d8a354727 service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:52.496187 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb99f9a-22da-47f4-aae4-158d8a354727 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:52.497562 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98703153-e0ff-44a9-a88f-331c49fd3ffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=56535a702f4344aa8a58f11406acf4c9)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:52.499499 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98703153-e0ff-44a9-a88f-331c49fd3ffd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:52.502591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-0bb99f9a-22da-47f4-aae4-158d8a354727 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:52.515303 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb99f9a-22da-47f4-aae4-158d8a354727 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:52.516951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb99f9a-22da-47f4-aae4-158d8a354727 service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:52.529346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1371/2721] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:52.567840 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-98703153-e0ff-44a9-a88f-331c49fd3ffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56535a702f4344aa8a58f11406acf4c9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.569403 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-98703153-e0ff-44a9-a88f-331c49fd3ffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56535a702f4344aa8a58f11406acf4c9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.581896 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-98703153-e0ff-44a9-a88f-331c49fd3ffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56535a702f4344aa8a58f11406acf4c9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.589181 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-98703153-e0ff-44a9-a88f-331c49fd3ffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56535a702f4344aa8a58f11406acf4c9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.589821 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-98703153-e0ff-44a9-a88f-331c49fd3ffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56535a702f4344aa8a58f11406acf4c9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.602555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1351/2722] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:37:52 2022] DELETE /identity/v3/users/56535a702f4344aa8a58f11406acf4c9 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:37:52.777541 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb99f9a-22da-47f4-aae4-158d8a354727 service placement] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:52.789561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb99f9a-22da-47f4-aae4-158d8a354727 service placement] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:52.847681 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e97775-1853-47b6-a102-18b0df1bb84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:52.848308 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e97775-1853-47b6-a102-18b0df1bb84b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:52.848589 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e97775-1853-47b6-a102-18b0df1bb84b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:52.848862 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e97775-1853-47b6-a102-18b0df1bb84b admin admin] PATH_INFO: `/v3/projects/7e39c6ca75a3466d93195fe245509bc9` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:52.849972 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e97775-1853-47b6-a102-18b0df1bb84b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e39c6ca75a3466d93195fe245509bc9)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:52.851969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e97775-1853-47b6-a102-18b0df1bb84b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:52.898816 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b3e97775-1853-47b6-a102-18b0df1bb84b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e39c6ca75a3466d93195fe245509bc9', 'request_id': 'req-b3e97775-1853-47b6-a102-18b0df1bb84b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.904853 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b3e97775-1853-47b6-a102-18b0df1bb84b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e39c6ca75a3466d93195fe245509bc9', 'request_id': 'req-b3e97775-1853-47b6-a102-18b0df1bb84b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.912272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b3e97775-1853-47b6-a102-18b0df1bb84b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e39c6ca75a3466d93195fe245509bc9', 'request_id': 'req-b3e97775-1853-47b6-a102-18b0df1bb84b'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.922691 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b3e97775-1853-47b6-a102-18b0df1bb84b admin admin] The token cache is being invalidate because project 7e39c6ca75a3466d93195fe245509bc9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:37:52.922973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b3e97775-1853-47b6-a102-18b0df1bb84b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.923586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-b3e97775-1853-47b6-a102-18b0df1bb84b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:37:52.925052 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1372/2723] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:37:52 2022] DELETE /identity/v3/projects/7e39c6ca75a3466d93195fe245509bc9 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:52.929916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3625ff5-6ee1-4a50-a282-1f13b7d160c7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:52.930249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3625ff5-6ee1-4a50-a282-1f13b7d160c7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:52.930517 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3625ff5-6ee1-4a50-a282-1f13b7d160c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:52.951289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c3625ff5-6ee1-4a50-a282-1f13b7d160c7 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:52.952498 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c3625ff5-6ee1-4a50-a282-1f13b7d160c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:52.969607 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:53.013249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1352/2724] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:37:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:53.020963 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9d05e-20a1-4fdf-a9f6-89208b8d3f90 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:53.021249 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9d05e-20a1-4fdf-a9f6-89208b8d3f90 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:53.021524 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9d05e-20a1-4fdf-a9f6-89208b8d3f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:53.041872 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cec9d05e-20a1-4fdf-a9f6-89208b8d3f90 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:53.043108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cec9d05e-20a1-4fdf-a9f6-89208b8d3f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:53.051598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1373/2725] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:53.055876 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3625ff5-6ee1-4a50-a282-1f13b7d160c7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:53.066239 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3625ff5-6ee1-4a50-a282-1f13b7d160c7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:53.070788 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051c4b1-acf9-4b60-a90d-a8a010ad0323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:53.071382 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051c4b1-acf9-4b60-a90d-a8a010ad0323 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:53.071657 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051c4b1-acf9-4b60-a90d-a8a010ad0323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:53.071922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051c4b1-acf9-4b60-a90d-a8a010ad0323 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:53.072558 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051c4b1-acf9-4b60-a90d-a8a010ad0323 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:37:53.072854 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051c4b1-acf9-4b60-a90d-a8a010ad0323 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:53.074495 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051c4b1-acf9-4b60-a90d-a8a010ad0323 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:53.086312 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1353/2726] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:37:53.090802 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9d05e-20a1-4fdf-a9f6-89208b8d3f90 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:53.100106 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9d05e-20a1-4fdf-a9f6-89208b8d3f90 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:53.104634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a7dc1-56fa-41c5-abae-23e0f59cf4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:53.105218 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a7dc1-56fa-41c5-abae-23e0f59cf4cc admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:53.105492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a7dc1-56fa-41c5-abae-23e0f59cf4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:53.105761 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994a7dc1-56fa-41c5-abae-23e0f59cf4cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:53.106410 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a7dc1-56fa-41c5-abae-23e0f59cf4cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:53.108304 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994a7dc1-56fa-41c5-abae-23e0f59cf4cc admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:53.119234 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:53.137045 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1374/2727] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:37:53 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:37:53.141535 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051c4b1-acf9-4b60-a90d-a8a010ad0323 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:53.151375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051c4b1-acf9-4b60-a90d-a8a010ad0323 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:53.207424 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0dd61a-b47f-4285-ae1e-0fece503555e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:53.208017 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dd61a-b47f-4285-ae1e-0fece503555e admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:53.208292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dd61a-b47f-4285-ae1e-0fece503555e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:53.208560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dd61a-b47f-4285-ae1e-0fece503555e admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:53.209176 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0dd61a-b47f-4285-ae1e-0fece503555e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:53.211088 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0dd61a-b47f-4285-ae1e-0fece503555e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:53.232200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1354/2728] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:37:53 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:37:53.236416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a7dc1-56fa-41c5-abae-23e0f59cf4cc admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:53.247252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994a7dc1-56fa-41c5-abae-23e0f59cf4cc admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:53.252084 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddcf99-33f8-4929-9c9f-9a98228927ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:53.252674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddcf99-33f8-4929-9c9f-9a98228927ba admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:53.252949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddcf99-33f8-4929-9c9f-9a98228927ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:53.253235 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddcf99-33f8-4929-9c9f-9a98228927ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:53.253897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cddcf99-33f8-4929-9c9f-9a98228927ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:53.255511 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cddcf99-33f8-4929-9c9f-9a98228927ba admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:53.265363 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1375/2729] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:53.269200 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0dd61a-b47f-4285-ae1e-0fece503555e admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:53.279395 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0dd61a-b47f-4285-ae1e-0fece503555e admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:53.284037 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0deccd-5b79-4765-aab1-b477bc4377e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:53.284631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0deccd-5b79-4765-aab1-b477bc4377e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:53.284912 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0deccd-5b79-4765-aab1-b477bc4377e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:53.285182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0deccd-5b79-4765-aab1-b477bc4377e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:53.286106 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0deccd-5b79-4765-aab1-b477bc4377e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:53.287716 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0deccd-5b79-4765-aab1-b477bc4377e6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:53.299293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1355/2730] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:37:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:37:53.303390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddcf99-33f8-4929-9c9f-9a98228927ba admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:53.314122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddcf99-33f8-4929-9c9f-9a98228927ba admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:53.318731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2fa3b-f06a-464c-b2cd-c28d1d8d5850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:53.319367 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2fa3b-f06a-464c-b2cd-c28d1d8d5850 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:53.319642 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2fa3b-f06a-464c-b2cd-c28d1d8d5850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:53.319912 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2fa3b-f06a-464c-b2cd-c28d1d8d5850 admin admin] PATH_INFO: `/v3/projects/2839782e51514ea698132e8a290b9a20/users/93b9cabb6bcb4141937ea814e0123bf0/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:53.334121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2fa3b-f06a-464c-b2cd-c28d1d8d5850 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2839782e51514ea698132e8a290b9a20, user_id=93b9cabb6bcb4141937ea814e0123bf0, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:53.336772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc2fa3b-f06a-464c-b2cd-c28d1d8d5850 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:53.349739 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1376/2731] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:37:53 2022] PUT /identity/v3/projects/2839782e51514ea698132e8a290b9a20/users/93b9cabb6bcb4141937ea814e0123bf0/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:37:53.354864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdf2ff-e3fd-4355-abb6-bc0e0204f7ae None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:53.355204 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdf2ff-e3fd-4355-abb6-bc0e0204f7ae None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:53.355482 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacdf2ff-e3fd-4355-abb6-bc0e0204f7ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:53.375716 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-aacdf2ff-e3fd-4355-abb6-bc0e0204f7ae None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:53.376916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-aacdf2ff-e3fd-4355-abb6-bc0e0204f7ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:53.435440 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1356/2732] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:37:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:53.444940 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2fa3b-f06a-464c-b2cd-c28d1d8d5850 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:53.446652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9bc2fa3b-f06a-464c-b2cd-c28d1d8d5850 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:53.502419 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2fa3b-f06a-464c-b2cd-c28d1d8d5850 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:53.506090 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b31dba-d179-49f6-a5fa-567fcc278111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:53.506728 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b31dba-d179-49f6-a5fa-567fcc278111 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:53.507006 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b31dba-d179-49f6-a5fa-567fcc278111 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:53.507273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b31dba-d179-49f6-a5fa-567fcc278111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:53.516403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b31dba-d179-49f6-a5fa-567fcc278111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:53.517769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b31dba-d179-49f6-a5fa-567fcc278111 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:53.527252 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1377/2733] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:57.530966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d476065-20df-4a28-b008-5b8cbd753c23 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:57.531450 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d476065-20df-4a28-b008-5b8cbd753c23 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:57.531549 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d476065-20df-4a28-b008-5b8cbd753c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:57.564410 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2d476065-20df-4a28-b008-5b8cbd753c23 None None] MFA Rules not processed for user `93b9cabb6bcb4141937ea814e0123bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:57.565676 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2d476065-20df-4a28-b008-5b8cbd753c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:57.583407 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:57.633902 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1357/2734] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:37:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:57.649709 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b31dba-d179-49f6-a5fa-567fcc278111 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:57.651243 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f1b31dba-d179-49f6-a5fa-567fcc278111 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:57.727185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b31dba-d179-49f6-a5fa-567fcc278111 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:57.732643 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61c1b9-204b-4444-8ba6-6a3f39e9a5a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:57.733236 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61c1b9-204b-4444-8ba6-6a3f39e9a5a4 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:57.733513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61c1b9-204b-4444-8ba6-6a3f39e9a5a4 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:57.733782 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af61c1b9-204b-4444-8ba6-6a3f39e9a5a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:57.741475 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61c1b9-204b-4444-8ba6-6a3f39e9a5a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:57.743100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af61c1b9-204b-4444-8ba6-6a3f39e9a5a4 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:57.752087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1378/2735] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:37:59.079457 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfd36c-cc65-4be2-a0b5-c4100c758b36 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:59.079966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfd36c-cc65-4be2-a0b5-c4100c758b36 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:59.080120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cfd36c-cc65-4be2-a0b5-c4100c758b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:59.099094 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-a9cfd36c-cc65-4be2-a0b5-c4100c758b36 None None] MFA Rules not processed for user `93b9cabb6bcb4141937ea814e0123bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:37:59.100406 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a9cfd36c-cc65-4be2-a0b5-c4100c758b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:59.108461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1358/2736] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:37:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:37:59.791642 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61c1b9-204b-4444-8ba6-6a3f39e9a5a4 service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:37:59.793701 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-af61c1b9-204b-4444-8ba6-6a3f39e9a5a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:59.866838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:37:59.895444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af61c1b9-204b-4444-8ba6-6a3f39e9a5a4 service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:37:59.900500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86e956-6be4-4f35-97a4-44eb15b68f9c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:37:59.901117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86e956-6be4-4f35-97a4-44eb15b68f9c service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:37:59.901399 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86e956-6be4-4f35-97a4-44eb15b68f9c service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:37:59.901674 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb86e956-6be4-4f35-97a4-44eb15b68f9c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:37:59.904216 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fb86e956-6be4-4f35-97a4-44eb15b68f9c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:37:59.917087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86e956-6be4-4f35-97a4-44eb15b68f9c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:37:59.918771 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb86e956-6be4-4f35-97a4-44eb15b68f9c service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:37:59.929992 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1379/2737] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:37:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:38:01.701345 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cfd36c-cc65-4be2-a0b5-c4100c758b36 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:01.702837 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a9cfd36c-cc65-4be2-a0b5-c4100c758b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:01.716263 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cfd36c-cc65-4be2-a0b5-c4100c758b36 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:01.721638 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44816cc-47a0-43d3-8e8e-c308964ec6a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:01.722526 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44816cc-47a0-43d3-8e8e-c308964ec6a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:01.722873 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44816cc-47a0-43d3-8e8e-c308964ec6a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:01.723204 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44816cc-47a0-43d3-8e8e-c308964ec6a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:01.725120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d44816cc-47a0-43d3-8e8e-c308964ec6a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:01.735422 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44816cc-47a0-43d3-8e8e-c308964ec6a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:01.736939 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44816cc-47a0-43d3-8e8e-c308964ec6a6 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:01.744891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1359/2738] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:03.428633 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86e956-6be4-4f35-97a4-44eb15b68f9c service placement] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:03.438664 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb86e956-6be4-4f35-97a4-44eb15b68f9c service placement] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:03.442195 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c440a8-0ec1-4d22-a025-8157545ae0e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:03.442806 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c440a8-0ec1-4d22-a025-8157545ae0e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:03.443082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c440a8-0ec1-4d22-a025-8157545ae0e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:03.443347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c440a8-0ec1-4d22-a025-8157545ae0e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:03.445019 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b0c440a8-0ec1-4d22-a025-8157545ae0e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:03.509639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c440a8-0ec1-4d22-a025-8157545ae0e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:03.511100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c440a8-0ec1-4d22-a025-8157545ae0e7 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:03.522990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1380/2739] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:04.691102 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44816cc-47a0-43d3-8e8e-c308964ec6a6 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:04.692635 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d44816cc-47a0-43d3-8e8e-c308964ec6a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:04.710272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44816cc-47a0-43d3-8e8e-c308964ec6a6 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:04.716857 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db390f11-ffae-40de-9721-b36f2292141e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:04.717637 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db390f11-ffae-40de-9721-b36f2292141e service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:04.717980 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db390f11-ffae-40de-9721-b36f2292141e service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:04.718344 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db390f11-ffae-40de-9721-b36f2292141e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:04.720166 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-db390f11-ffae-40de-9721-b36f2292141e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:04.733014 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db390f11-ffae-40de-9721-b36f2292141e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:04.734726 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db390f11-ffae-40de-9721-b36f2292141e service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:04.746158 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1360/2740] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:38:07.932431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c440a8-0ec1-4d22-a025-8157545ae0e7 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:07.947813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c440a8-0ec1-4d22-a025-8157545ae0e7 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:07.953690 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ebb42-e92f-408e-b2cf-be1f90040d41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:07.954359 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ebb42-e92f-408e-b2cf-be1f90040d41 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:07.954638 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ebb42-e92f-408e-b2cf-be1f90040d41 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:07.954905 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54ebb42-e92f-408e-b2cf-be1f90040d41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:07.956634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d54ebb42-e92f-408e-b2cf-be1f90040d41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:07.969897 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ebb42-e92f-408e-b2cf-be1f90040d41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:07.971523 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54ebb42-e92f-408e-b2cf-be1f90040d41 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:07.983151 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1381/2741] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:08.562512 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db390f11-ffae-40de-9721-b36f2292141e service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:08.571930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db390f11-ffae-40de-9721-b36f2292141e service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:08.576070 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f9038-c6d9-4f94-8f48-1bf4f92a6304 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:08.576665 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f9038-c6d9-4f94-8f48-1bf4f92a6304 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:08.576938 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f9038-c6d9-4f94-8f48-1bf4f92a6304 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:08.577203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f9038-c6d9-4f94-8f48-1bf4f92a6304 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:08.578915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f83f9038-c6d9-4f94-8f48-1bf4f92a6304 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:08.590729 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f9038-c6d9-4f94-8f48-1bf4f92a6304 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:08.592236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f9038-c6d9-4f94-8f48-1bf4f92a6304 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:08.602554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1361/2742] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:38:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:38:15.241358 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ebb42-e92f-408e-b2cf-be1f90040d41 service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:15.243029 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d54ebb42-e92f-408e-b2cf-be1f90040d41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:15.258526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54ebb42-e92f-408e-b2cf-be1f90040d41 service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:15.263117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e0270-87a1-4e10-a7e2-763309989bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:15.263849 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e0270-87a1-4e10-a7e2-763309989bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:15.264183 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e0270-87a1-4e10-a7e2-763309989bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:15.264519 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e0270-87a1-4e10-a7e2-763309989bca admin admin] PATH_INFO: `/v3/users/63b945e0d46946bfad2d89a0fd15fd8c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:15.265884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e0270-87a1-4e10-a7e2-763309989bca admin admin] RBAC: Authorizing `identity:delete_user(user_id=63b945e0d46946bfad2d89a0fd15fd8c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:15.268095 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e0270-87a1-4e10-a7e2-763309989bca admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:15.318614 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-391e0270-87a1-4e10-a7e2-763309989bca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63b945e0d46946bfad2d89a0fd15fd8c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:15.329034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-391e0270-87a1-4e10-a7e2-763309989bca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63b945e0d46946bfad2d89a0fd15fd8c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:15.342633 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-391e0270-87a1-4e10-a7e2-763309989bca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63b945e0d46946bfad2d89a0fd15fd8c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:15.343560 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-391e0270-87a1-4e10-a7e2-763309989bca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63b945e0d46946bfad2d89a0fd15fd8c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:15.353313 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-391e0270-87a1-4e10-a7e2-763309989bca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63b945e0d46946bfad2d89a0fd15fd8c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:15.360186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1382/2743] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:15 2022] DELETE /identity/v3/users/63b945e0d46946bfad2d89a0fd15fd8c => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:15.541093 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f9038-c6d9-4f94-8f48-1bf4f92a6304 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:15.553812 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f9038-c6d9-4f94-8f48-1bf4f92a6304 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:15.611891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e48b0-771b-493c-9315-89091e6775c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:15.612514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e48b0-771b-493c-9315-89091e6775c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:15.612789 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e48b0-771b-493c-9315-89091e6775c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:15.613056 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e48b0-771b-493c-9315-89091e6775c7 admin admin] PATH_INFO: `/v3/projects/68476a44d7ee4d1d9fbf3c7516885edf` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:15.614143 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e48b0-771b-493c-9315-89091e6775c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68476a44d7ee4d1d9fbf3c7516885edf)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:15.616055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e48b0-771b-493c-9315-89091e6775c7 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:15.664374 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f35e48b0-771b-493c-9315-89091e6775c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68476a44d7ee4d1d9fbf3c7516885edf', 'request_id': 'req-f35e48b0-771b-493c-9315-89091e6775c7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:15.669966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f35e48b0-771b-493c-9315-89091e6775c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68476a44d7ee4d1d9fbf3c7516885edf', 'request_id': 'req-f35e48b0-771b-493c-9315-89091e6775c7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:15.678139 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f35e48b0-771b-493c-9315-89091e6775c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68476a44d7ee4d1d9fbf3c7516885edf', 'request_id': 'req-f35e48b0-771b-493c-9315-89091e6775c7'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:15.688977 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f35e48b0-771b-493c-9315-89091e6775c7 admin admin] The token cache is being invalidate because project 68476a44d7ee4d1d9fbf3c7516885edf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:15.689260 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f35e48b0-771b-493c-9315-89091e6775c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:15.690057 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-f35e48b0-771b-493c-9315-89091e6775c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:15.691649 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1362/2744] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:15 2022] DELETE /identity/v3/projects/68476a44d7ee4d1d9fbf3c7516885edf => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:15.695707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e0270-87a1-4e10-a7e2-763309989bca admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:15.697137 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-391e0270-87a1-4e10-a7e2-763309989bca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:15.710977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e0270-87a1-4e10-a7e2-763309989bca admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:15.740018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:15.769463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70db720-09b5-4557-9952-7e3e6adb9f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:15.770109 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70db720-09b5-4557-9952-7e3e6adb9f70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:15.770406 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70db720-09b5-4557-9952-7e3e6adb9f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:15.770675 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70db720-09b5-4557-9952-7e3e6adb9f70 admin admin] PATH_INFO: `/v3/users/88271ab2d084420ab6d57f5d63c0b12f` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:15.771789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70db720-09b5-4557-9952-7e3e6adb9f70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88271ab2d084420ab6d57f5d63c0b12f)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:15.773730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70db720-09b5-4557-9952-7e3e6adb9f70 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:15.820245 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c70db720-09b5-4557-9952-7e3e6adb9f70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88271ab2d084420ab6d57f5d63c0b12f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:15.828908 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c70db720-09b5-4557-9952-7e3e6adb9f70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88271ab2d084420ab6d57f5d63c0b12f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:15.840889 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c70db720-09b5-4557-9952-7e3e6adb9f70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88271ab2d084420ab6d57f5d63c0b12f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:15.841540 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c70db720-09b5-4557-9952-7e3e6adb9f70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88271ab2d084420ab6d57f5d63c0b12f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:15.848445 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-c70db720-09b5-4557-9952-7e3e6adb9f70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88271ab2d084420ab6d57f5d63c0b12f'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:15.854275 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1383/2745] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:38:15 2022] DELETE /identity/v3/users/88271ab2d084420ab6d57f5d63c0b12f => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:16.025515 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e48b0-771b-493c-9315-89091e6775c7 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:38:16.026954 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-f35e48b0-771b-493c-9315-89091e6775c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:38:16.035575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:16.042815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e48b0-771b-493c-9315-89091e6775c7 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:38:16.100677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:38:16.101309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:38:16.101588 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:38:16.101859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf admin admin] PATH_INFO: `/v3/projects/5939401400f5466ba706e2df4f20c72f` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:38:16.103003 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=5939401400f5466ba706e2df4f20c72f)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:38:16.104893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:38:16.151442 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5939401400f5466ba706e2df4f20c72f', 'request_id': 'req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:16.156828 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5939401400f5466ba706e2df4f20c72f', 'request_id': 'req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:16.163998 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5939401400f5466ba706e2df4f20c72f', 'request_id': 'req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:16.175289 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf admin admin] The token cache is being invalidate because project 5939401400f5466ba706e2df4f20c72f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:38:16.175581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:16.176214 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:38:16.177978 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1363/2746] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:38:16 2022] DELETE /identity/v3/projects/5939401400f5466ba706e2df4f20c72f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:38:25.850910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=39,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:38:26.171987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=50,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:39:46.141163 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70db720-09b5-4557-9952-7e3e6adb9f70 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:39:46.142909 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c70db720-09b5-4557-9952-7e3e6adb9f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:46.211353 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70db720-09b5-4557-9952-7e3e6adb9f70 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:39:46.271478 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1dfb3-2520-476d-99fb-e273466f9299 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:39:46.272085 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1dfb3-2520-476d-99fb-e273466f9299 service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:39:46.272363 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1dfb3-2520-476d-99fb-e273466f9299 service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:39:46.272639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1dfb3-2520-476d-99fb-e273466f9299 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:39:46.274411 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7ea1dfb3-2520-476d-99fb-e273466f9299 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:39:46.286154 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1dfb3-2520-476d-99fb-e273466f9299 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:39:46.288070 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea1dfb3-2520-476d-99fb-e273466f9299 service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:39:46.298549 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1384/2747] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:39:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:39:56.294316 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=58 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:42:13.591935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:13.593425 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:13.651223 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e58b79-4327-4157-9fe0-ab8fd5f5f0bf admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:13.654911 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2add5a-47eb-447e-a41e-c9e2d6f05a7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:13.655538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2add5a-47eb-447e-a41e-c9e2d6f05a7a service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:13.655805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2add5a-47eb-447e-a41e-c9e2d6f05a7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:13.656075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2add5a-47eb-447e-a41e-c9e2d6f05a7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:13.657843 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bb2add5a-47eb-447e-a41e-c9e2d6f05a7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:13.732792 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2add5a-47eb-447e-a41e-c9e2d6f05a7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:13.734198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2add5a-47eb-447e-a41e-c9e2d6f05a7a service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:13.745564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1364/2748] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:14.928309 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1dfb3-2520-476d-99fb-e273466f9299 service placement] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:14.929741 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7ea1dfb3-2520-476d-99fb-e273466f9299 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:14.941107 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1dfb3-2520-476d-99fb-e273466f9299 service placement] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:14.945166 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8934a28f-4e2a-4ee7-b83d-415848044bd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:14.945755 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934a28f-4e2a-4ee7-b83d-415848044bd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:14.946027 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934a28f-4e2a-4ee7-b83d-415848044bd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:14.946334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8934a28f-4e2a-4ee7-b83d-415848044bd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:14.947989 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8934a28f-4e2a-4ee7-b83d-415848044bd6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:14.961477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934a28f-4e2a-4ee7-b83d-415848044bd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:14.962917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8934a28f-4e2a-4ee7-b83d-415848044bd6 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:14.973744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1385/2749] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:16.702987 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2add5a-47eb-447e-a41e-c9e2d6f05a7a service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:16.704712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-bb2add5a-47eb-447e-a41e-c9e2d6f05a7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:16.721502 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2add5a-47eb-447e-a41e-c9e2d6f05a7a service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:16.728016 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af5dc9-3b6b-4a16-8919-3d52ec72caf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:16.728680 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5dc9-3b6b-4a16-8919-3d52ec72caf4 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:16.729098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5dc9-3b6b-4a16-8919-3d52ec72caf4 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:16.729381 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af5dc9-3b6b-4a16-8919-3d52ec72caf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:16.731292 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-06af5dc9-3b6b-4a16-8919-3d52ec72caf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:16.748062 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af5dc9-3b6b-4a16-8919-3d52ec72caf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:16.749769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af5dc9-3b6b-4a16-8919-3d52ec72caf4 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:16.763144 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1365/2750] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:20.275743 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8b005-4108-473b-96d6-5fb3bd574685 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:20.276291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8b005-4108-473b-96d6-5fb3bd574685 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:20.276291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec8b005-4108-473b-96d6-5fb3bd574685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:20.295466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8ec8b005-4108-473b-96d6-5fb3bd574685 None None] MFA Rules not processed for user `93b9cabb6bcb4141937ea814e0123bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:42:20.296652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8ec8b005-4108-473b-96d6-5fb3bd574685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:20.351720 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1386/2751] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:42:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:42:20.654658 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af5dc9-3b6b-4a16-8919-3d52ec72caf4 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:42:20.663892 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af5dc9-3b6b-4a16-8919-3d52ec72caf4 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:42:20.667712 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4557309-0808-4a16-a825-a15cb792d9ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:42:20.668298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4557309-0808-4a16-a825-a15cb792d9ab service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:42:20.668575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4557309-0808-4a16-a825-a15cb792d9ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:42:20.668842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4557309-0808-4a16-a825-a15cb792d9ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:42:20.670577 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e4557309-0808-4a16-a825-a15cb792d9ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:42:20.681363 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4557309-0808-4a16-a825-a15cb792d9ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:42:20.682822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4557309-0808-4a16-a825-a15cb792d9ab service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:42:20.690827 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1366/2752] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:42:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:42:30.345149 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=31 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:42:30.687158 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=66 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:37.489354 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8b005-4108-473b-96d6-5fb3bd574685 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:37.503965 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec8b005-4108-473b-96d6-5fb3bd574685 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:37.509409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968a0b6-df00-4fff-8ad5-ece3f36e571d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:37.510004 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968a0b6-df00-4fff-8ad5-ece3f36e571d service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:37.510312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968a0b6-df00-4fff-8ad5-ece3f36e571d service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:37.510913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a968a0b6-df00-4fff-8ad5-ece3f36e571d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:37.512619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a968a0b6-df00-4fff-8ad5-ece3f36e571d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:37.523508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968a0b6-df00-4fff-8ad5-ece3f36e571d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:37.525064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a968a0b6-df00-4fff-8ad5-ece3f36e571d service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:37.534425 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1387/2753] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:43:37 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:43:47.530009 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:43:48.338916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4557309-0808-4a16-a825-a15cb792d9ab service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:43:48.349439 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4557309-0808-4a16-a825-a15cb792d9ab service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:43:48.354918 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29972bff-6b80-4624-b7c2-da3180071590 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:43:48.355523 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29972bff-6b80-4624-b7c2-da3180071590 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:43:48.355796 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29972bff-6b80-4624-b7c2-da3180071590 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:43:48.356065 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29972bff-6b80-4624-b7c2-da3180071590 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:43:48.357764 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-29972bff-6b80-4624-b7c2-da3180071590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:43:48.370901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29972bff-6b80-4624-b7c2-da3180071590 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:43:48.372963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29972bff-6b80-4624-b7c2-da3180071590 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:43:48.383084 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1367/2754] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:43:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:43:58.379000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:07.157479 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968a0b6-df00-4fff-8ad5-ece3f36e571d service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:07.167808 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a968a0b6-df00-4fff-8ad5-ece3f36e571d service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:07.171416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78bf39-1900-44e7-b78d-16ed3ac413c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:07.172015 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78bf39-1900-44e7-b78d-16ed3ac413c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:07.172294 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78bf39-1900-44e7-b78d-16ed3ac413c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:07.172567 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d78bf39-1900-44e7-b78d-16ed3ac413c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:07.174341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6d78bf39-1900-44e7-b78d-16ed3ac413c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:07.236808 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78bf39-1900-44e7-b78d-16ed3ac413c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:07.238283 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d78bf39-1900-44e7-b78d-16ed3ac413c7 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:07.248735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1388/2755] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:10.147677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29972bff-6b80-4624-b7c2-da3180071590 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:10.149218 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-29972bff-6b80-4624-b7c2-da3180071590 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:10.164873 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29972bff-6b80-4624-b7c2-da3180071590 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:10.170520 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3f041-7dfb-4cb4-853e-5411c9c22316 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:10.171127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3f041-7dfb-4cb4-853e-5411c9c22316 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:10.171407 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3f041-7dfb-4cb4-853e-5411c9c22316 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:10.171682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef3f041-7dfb-4cb4-853e-5411c9c22316 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:10.173463 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fef3f041-7dfb-4cb4-853e-5411c9c22316 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:10.185871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3f041-7dfb-4cb4-853e-5411c9c22316 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:10.188649 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef3f041-7dfb-4cb4-853e-5411c9c22316 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:10.199223 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1368/2756] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:10.480323 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78bf39-1900-44e7-b78d-16ed3ac413c7 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:10.484946 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-6d78bf39-1900-44e7-b78d-16ed3ac413c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:10.496626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d78bf39-1900-44e7-b78d-16ed3ac413c7 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:10.501315 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577c121e-fb73-4cea-8441-1eb201ff954f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:10.501965 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c121e-fb73-4cea-8441-1eb201ff954f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:10.502270 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c121e-fb73-4cea-8441-1eb201ff954f admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:10.502679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577c121e-fb73-4cea-8441-1eb201ff954f admin admin] PATH_INFO: `/v3/users/93b9cabb6bcb4141937ea814e0123bf0` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:10.503823 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c121e-fb73-4cea-8441-1eb201ff954f admin admin] RBAC: Authorizing `identity:delete_user(user_id=93b9cabb6bcb4141937ea814e0123bf0)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:10.506187 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577c121e-fb73-4cea-8441-1eb201ff954f admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:10.558800 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-577c121e-fb73-4cea-8441-1eb201ff954f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93b9cabb6bcb4141937ea814e0123bf0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.566516 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-577c121e-fb73-4cea-8441-1eb201ff954f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93b9cabb6bcb4141937ea814e0123bf0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.580550 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-577c121e-fb73-4cea-8441-1eb201ff954f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93b9cabb6bcb4141937ea814e0123bf0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.581686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-577c121e-fb73-4cea-8441-1eb201ff954f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93b9cabb6bcb4141937ea814e0123bf0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.589100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-577c121e-fb73-4cea-8441-1eb201ff954f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93b9cabb6bcb4141937ea814e0123bf0'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.595005 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1389/2757] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:44:10 2022] DELETE /identity/v3/users/93b9cabb6bcb4141937ea814e0123bf0 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:10.770613 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3f041-7dfb-4cb4-853e-5411c9c22316 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:10.783432 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef3f041-7dfb-4cb4-853e-5411c9c22316 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:10.846620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a6fbc4-0cbb-478c-8824-a507c10f9174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:10.847251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6fbc4-0cbb-478c-8824-a507c10f9174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:10.847536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6fbc4-0cbb-478c-8824-a507c10f9174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:10.847820 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6fbc4-0cbb-478c-8824-a507c10f9174 admin admin] PATH_INFO: `/v3/projects/2839782e51514ea698132e8a290b9a20` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:10.848935 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a6fbc4-0cbb-478c-8824-a507c10f9174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2839782e51514ea698132e8a290b9a20)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:10.851018 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a6fbc4-0cbb-478c-8824-a507c10f9174 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:10.897805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-79a6fbc4-0cbb-478c-8824-a507c10f9174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2839782e51514ea698132e8a290b9a20', 'request_id': 'req-79a6fbc4-0cbb-478c-8824-a507c10f9174'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.903195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-79a6fbc4-0cbb-478c-8824-a507c10f9174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2839782e51514ea698132e8a290b9a20', 'request_id': 'req-79a6fbc4-0cbb-478c-8824-a507c10f9174'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.910341 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-79a6fbc4-0cbb-478c-8824-a507c10f9174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2839782e51514ea698132e8a290b9a20', 'request_id': 'req-79a6fbc4-0cbb-478c-8824-a507c10f9174'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.921860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-79a6fbc4-0cbb-478c-8824-a507c10f9174 admin admin] The token cache is being invalidate because project 2839782e51514ea698132e8a290b9a20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:44:10.922147 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-79a6fbc4-0cbb-478c-8824-a507c10f9174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.922827 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-79a6fbc4-0cbb-478c-8824-a507c10f9174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:44:10.924286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1369/2758] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:44:10 2022] DELETE /identity/v3/projects/2839782e51514ea698132e8a290b9a20 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:10.929442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca287d-0241-4301-80e9-10a0a4348eee None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:10.929754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca287d-0241-4301-80e9-10a0a4348eee None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:10.930127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca287d-0241-4301-80e9-10a0a4348eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:10.950464 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-c8ca287d-0241-4301-80e9-10a0a4348eee None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:10.951775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c8ca287d-0241-4301-80e9-10a0a4348eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:11.018180 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1390/2759] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:44:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:11.027584 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1e964-9572-4e73-bd0e-7e6b84a370d7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:11.028038 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1e964-9572-4e73-bd0e-7e6b84a370d7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:11.028214 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae1e964-9572-4e73-bd0e-7e6b84a370d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:11.050159 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2ae1e964-9572-4e73-bd0e-7e6b84a370d7 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:11.051409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2ae1e964-9572-4e73-bd0e-7e6b84a370d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:11.059323 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1370/2760] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:11.063801 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca287d-0241-4301-80e9-10a0a4348eee None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:11.074269 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca287d-0241-4301-80e9-10a0a4348eee None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:11.078979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa51738-6d33-45df-a44f-4b4176c89325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:11.079587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa51738-6d33-45df-a44f-4b4176c89325 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:11.079879 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa51738-6d33-45df-a44f-4b4176c89325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:11.080132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa51738-6d33-45df-a44f-4b4176c89325 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:11.080784 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa51738-6d33-45df-a44f-4b4176c89325 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:44:11.081087 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa51738-6d33-45df-a44f-4b4176c89325 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:11.082718 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa51738-6d33-45df-a44f-4b4176c89325 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:11.084345 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:11.097534 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1391/2761] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:44:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:44:11.102019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1e964-9572-4e73-bd0e-7e6b84a370d7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:11.111776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae1e964-9572-4e73-bd0e-7e6b84a370d7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:11.116462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0909880-8ecb-49f4-a0ed-0e3055be7355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:11.117040 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0909880-8ecb-49f4-a0ed-0e3055be7355 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:11.117316 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0909880-8ecb-49f4-a0ed-0e3055be7355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:11.117587 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0909880-8ecb-49f4-a0ed-0e3055be7355 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:11.118202 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0909880-8ecb-49f4-a0ed-0e3055be7355 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:11.120286 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0909880-8ecb-49f4-a0ed-0e3055be7355 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:11.147608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1371/2762] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:44:11 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:44:11.152426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa51738-6d33-45df-a44f-4b4176c89325 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:11.163202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa51738-6d33-45df-a44f-4b4176c89325 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:11.219027 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26908393-6062-4ace-a079-70ae1ff75e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:11.219628 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26908393-6062-4ace-a079-70ae1ff75e29 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:11.219909 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26908393-6062-4ace-a079-70ae1ff75e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:11.220185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26908393-6062-4ace-a079-70ae1ff75e29 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:11.220815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26908393-6062-4ace-a079-70ae1ff75e29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:11.222758 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26908393-6062-4ace-a079-70ae1ff75e29 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:11.243661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1392/2763] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:44:11 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:44:11.247956 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0909880-8ecb-49f4-a0ed-0e3055be7355 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:11.258067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0909880-8ecb-49f4-a0ed-0e3055be7355 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:11.262647 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49427b39-3a85-42ce-b7b8-2e617b4fb257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:11.263235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49427b39-3a85-42ce-b7b8-2e617b4fb257 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:11.263513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49427b39-3a85-42ce-b7b8-2e617b4fb257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:11.263782 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49427b39-3a85-42ce-b7b8-2e617b4fb257 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:11.264414 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49427b39-3a85-42ce-b7b8-2e617b4fb257 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:11.265989 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49427b39-3a85-42ce-b7b8-2e617b4fb257 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:11.276314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1372/2764] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:11.280061 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26908393-6062-4ace-a079-70ae1ff75e29 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:11.290048 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26908393-6062-4ace-a079-70ae1ff75e29 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:11.296131 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899d9bdf-0de9-45ea-b6b4-75ec053fd267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:11.296717 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d9bdf-0de9-45ea-b6b4-75ec053fd267 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:11.296992 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d9bdf-0de9-45ea-b6b4-75ec053fd267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:11.297260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899d9bdf-0de9-45ea-b6b4-75ec053fd267 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:11.297896 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d9bdf-0de9-45ea-b6b4-75ec053fd267 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:11.299581 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899d9bdf-0de9-45ea-b6b4-75ec053fd267 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:11.310108 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1393/2765] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:44:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:44:11.314199 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49427b39-3a85-42ce-b7b8-2e617b4fb257 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:11.324293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49427b39-3a85-42ce-b7b8-2e617b4fb257 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:11.328891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590ea3b-4941-4116-81ef-1dfbe3ae0e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:11.329536 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590ea3b-4941-4116-81ef-1dfbe3ae0e39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:11.329866 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590ea3b-4941-4116-81ef-1dfbe3ae0e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:11.330245 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590ea3b-4941-4116-81ef-1dfbe3ae0e39 admin admin] PATH_INFO: `/v3/projects/c07806353e51499b99117ea363f78b0c/users/1f1ba27117d54534b9b423df00d43aa1/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:11.344937 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590ea3b-4941-4116-81ef-1dfbe3ae0e39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c07806353e51499b99117ea363f78b0c, user_id=1f1ba27117d54534b9b423df00d43aa1, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:11.347651 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0590ea3b-4941-4116-81ef-1dfbe3ae0e39 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:11.360829 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1373/2766] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:44:11 2022] PUT /identity/v3/projects/c07806353e51499b99117ea363f78b0c/users/1f1ba27117d54534b9b423df00d43aa1/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:44:11.365815 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f89dc6-f516-458d-9b02-6a103c7d8e40 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:11.366100 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f89dc6-f516-458d-9b02-6a103c7d8e40 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:11.366467 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f89dc6-f516-458d-9b02-6a103c7d8e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:11.385362 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-e9f89dc6-f516-458d-9b02-6a103c7d8e40 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:11.386593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-e9f89dc6-f516-458d-9b02-6a103c7d8e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:11.442979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1394/2767] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:11.453118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590ea3b-4941-4116-81ef-1dfbe3ae0e39 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:11.454834 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0590ea3b-4941-4116-81ef-1dfbe3ae0e39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:11.469426 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:11.513838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590ea3b-4941-4116-81ef-1dfbe3ae0e39 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:11.517496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99354340-6748-4fa3-b113-0838fce27511 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:11.518098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99354340-6748-4fa3-b113-0838fce27511 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:11.518405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99354340-6748-4fa3-b113-0838fce27511 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:11.518676 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99354340-6748-4fa3-b113-0838fce27511 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:11.528244 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99354340-6748-4fa3-b113-0838fce27511 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:11.529631 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99354340-6748-4fa3-b113-0838fce27511 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:11.539221 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1374/2768] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:15.736123 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfa72a-eee3-485a-b7f7-39844ec3bca1 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:15.736672 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfa72a-eee3-485a-b7f7-39844ec3bca1 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:15.736913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfa72a-eee3-485a-b7f7-39844ec3bca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:15.771212 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-72dfa72a-eee3-485a-b7f7-39844ec3bca1 None None] MFA Rules not processed for user `1f1ba27117d54534b9b423df00d43aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:15.772097 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-72dfa72a-eee3-485a-b7f7-39844ec3bca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:15.846255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1395/2769] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:44:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:15.860936 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99354340-6748-4fa3-b113-0838fce27511 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:15.862489 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-99354340-6748-4fa3-b113-0838fce27511 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:15.942584 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99354340-6748-4fa3-b113-0838fce27511 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:15.949089 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6372713-6228-4835-8d2a-51755cd3afd3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:15.949717 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6372713-6228-4835-8d2a-51755cd3afd3 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:15.949995 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6372713-6228-4835-8d2a-51755cd3afd3 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:15.950307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6372713-6228-4835-8d2a-51755cd3afd3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:15.959356 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6372713-6228-4835-8d2a-51755cd3afd3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:15.960962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6372713-6228-4835-8d2a-51755cd3afd3 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:15.970767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1375/2770] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:15 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:44:17.464106 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989d758-bf79-4370-a2a3-4a80c3612812 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:17.464606 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989d758-bf79-4370-a2a3-4a80c3612812 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:17.464707 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a989d758-bf79-4370-a2a3-4a80c3612812 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:17.472142 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:17.611931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a989d758-bf79-4370-a2a3-4a80c3612812 None None] MFA Rules not processed for user `1f1ba27117d54534b9b423df00d43aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:44:17.613172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a989d758-bf79-4370-a2a3-4a80c3612812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:17.625298 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1396/2771] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:44:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:44:18.381301 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6372713-6228-4835-8d2a-51755cd3afd3 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:18.388476 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b6372713-6228-4835-8d2a-51755cd3afd3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:18.538843 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6372713-6228-4835-8d2a-51755cd3afd3 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:18.545226 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64d5e1-3841-4e60-baab-699070ddcbe2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:18.545859 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64d5e1-3841-4e60-baab-699070ddcbe2 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:18.546134 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64d5e1-3841-4e60-baab-699070ddcbe2 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:18.546565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64d5e1-3841-4e60-baab-699070ddcbe2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:18.548368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fb64d5e1-3841-4e60-baab-699070ddcbe2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:18.563288 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64d5e1-3841-4e60-baab-699070ddcbe2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:18.564923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64d5e1-3841-4e60-baab-699070ddcbe2 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:18.577111 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1376/2772] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:27.606995 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:27.832783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a989d758-bf79-4370-a2a3-4a80c3612812 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:27.834380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a989d758-bf79-4370-a2a3-4a80c3612812 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:27.860543 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a989d758-bf79-4370-a2a3-4a80c3612812 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:27.867317 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b9de4-46a2-4734-acbb-fc477b76a0b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:27.867942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b9de4-46a2-4734-acbb-fc477b76a0b6 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:27.868218 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b9de4-46a2-4734-acbb-fc477b76a0b6 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:27.868483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b9de4-46a2-4734-acbb-fc477b76a0b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:27.870410 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d43b9de4-46a2-4734-acbb-fc477b76a0b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:27.887968 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b9de4-46a2-4734-acbb-fc477b76a0b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:27.889938 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43b9de4-46a2-4734-acbb-fc477b76a0b6 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:27.905312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1397/2773] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:44:28.572742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:37.899089 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:44:51.579891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64d5e1-3841-4e60-baab-699070ddcbe2 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:44:51.619422 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64d5e1-3841-4e60-baab-699070ddcbe2 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:44:51.624934 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ba15b-e5ca-4956-82ad-fd6aa8a7b4f1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:44:51.625529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ba15b-e5ca-4956-82ad-fd6aa8a7b4f1 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:44:51.625803 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ba15b-e5ca-4956-82ad-fd6aa8a7b4f1 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:44:51.626069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ba15b-e5ca-4956-82ad-fd6aa8a7b4f1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:44:51.627816 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-315ba15b-e5ca-4956-82ad-fd6aa8a7b4f1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:44:51.651509 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ba15b-e5ca-4956-82ad-fd6aa8a7b4f1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:44:51.653095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ba15b-e5ca-4956-82ad-fd6aa8a7b4f1 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:44:51.662814 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1377/2774] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:44:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:45:01.658593 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=24 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:45:57.651397 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe701d99-a2a9-4ef8-a80e-5d07eae9790c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:45:57.651951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe701d99-a2a9-4ef8-a80e-5d07eae9790c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:45:57.651951 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe701d99-a2a9-4ef8-a80e-5d07eae9790c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:45:57.680602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-fe701d99-a2a9-4ef8-a80e-5d07eae9790c None None] MFA Rules not processed for user `1f1ba27117d54534b9b423df00d43aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:45:57.681823 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fe701d99-a2a9-4ef8-a80e-5d07eae9790c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:45:57.692446 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1398/2775] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:45:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:45:58.006641 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ba15b-e5ca-4956-82ad-fd6aa8a7b4f1 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:45:58.043772 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ba15b-e5ca-4956-82ad-fd6aa8a7b4f1 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:45:58.047380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b5d55-ff4f-4a83-8741-6c4662d2fa03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:45:58.048000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b5d55-ff4f-4a83-8741-6c4662d2fa03 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:45:58.048324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b5d55-ff4f-4a83-8741-6c4662d2fa03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:45:58.049149 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3b5d55-ff4f-4a83-8741-6c4662d2fa03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:45:58.050912 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2b3b5d55-ff4f-4a83-8741-6c4662d2fa03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:45:58.071664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b5d55-ff4f-4a83-8741-6c4662d2fa03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:45:58.073007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3b5d55-ff4f-4a83-8741-6c4662d2fa03 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:45:58.080746 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1378/2776] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:45:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:46:07.690030 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=9 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:46:08.077244 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=23 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:47:05.285559 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628edea-2a31-45d3-ba09-a27e171a548c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:05.286109 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628edea-2a31-45d3-ba09-a27e171a548c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:05.286420 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628edea-2a31-45d3-ba09-a27e171a548c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:05.306072 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-7628edea-2a31-45d3-ba09-a27e171a548c None None] MFA Rules not processed for user `1f1ba27117d54534b9b423df00d43aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:47:05.307341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-7628edea-2a31-45d3-ba09-a27e171a548c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:05.314394 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1399/2777] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:47:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:47:15.178115 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b5d55-ff4f-4a83-8741-6c4662d2fa03 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:47:15.179690 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2b3b5d55-ff4f-4a83-8741-6c4662d2fa03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:15.190694 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3b5d55-ff4f-4a83-8741-6c4662d2fa03 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:47:15.194310 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0ca86-47d2-4670-904c-2c189ee0bac3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:47:15.194905 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0ca86-47d2-4670-904c-2c189ee0bac3 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:15.195194 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0ca86-47d2-4670-904c-2c189ee0bac3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:15.195444 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0ca86-47d2-4670-904c-2c189ee0bac3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:15.197183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-87d0ca86-47d2-4670-904c-2c189ee0bac3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:15.229879 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d0ca86-47d2-4670-904c-2c189ee0bac3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:47:15.231337 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d0ca86-47d2-4670-904c-2c189ee0bac3 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:47:15.252468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1379/2778] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:47:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:47:15.301159 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:47:18.768918 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7628edea-2a31-45d3-ba09-a27e171a548c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:47:18.782495 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7628edea-2a31-45d3-ba09-a27e171a548c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:47:18.788204 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56527eba-2a0d-488b-b314-a5a7b314fa43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:47:18.788842 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56527eba-2a0d-488b-b314-a5a7b314fa43 service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:47:18.789121 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56527eba-2a0d-488b-b314-a5a7b314fa43 service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:47:18.789393 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56527eba-2a0d-488b-b314-a5a7b314fa43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:47:18.791262 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-56527eba-2a0d-488b-b314-a5a7b314fa43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:47:18.804589 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56527eba-2a0d-488b-b314-a5a7b314fa43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:47:18.806165 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56527eba-2a0d-488b-b314-a5a7b314fa43 service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:47:18.817508 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1400/2779] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:47:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:47:25.249838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=18 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:47:28.812504 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:36.983199 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0ca86-47d2-4670-904c-2c189ee0bac3 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:36.999786 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0ca86-47d2-4670-904c-2c189ee0bac3 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:37.006156 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760edcc-6588-4776-a280-687b90333cb7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:37.007208 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760edcc-6588-4776-a280-687b90333cb7 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:37.007493 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760edcc-6588-4776-a280-687b90333cb7 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:37.007869 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0760edcc-6588-4776-a280-687b90333cb7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:37.009852 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-0760edcc-6588-4776-a280-687b90333cb7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:37.024334 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760edcc-6588-4776-a280-687b90333cb7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:37.025937 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0760edcc-6588-4776-a280-687b90333cb7 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:37.043266 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1380/2780] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:39.894291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcd5ac-af7f-4e0a-8e0a-115c9fc6ce82 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:39.894811 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcd5ac-af7f-4e0a-8e0a-115c9fc6ce82 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:39.894811 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbcd5ac-af7f-4e0a-8e0a-115c9fc6ce82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:39.913838 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-cfbcd5ac-af7f-4e0a-8e0a-115c9fc6ce82 None None] MFA Rules not processed for user `1f1ba27117d54534b9b423df00d43aa1`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:48:39.915076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cfbcd5ac-af7f-4e0a-8e0a-115c9fc6ce82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:39.922688 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1401/2781] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:48:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:48:40.229376 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760edcc-6588-4776-a280-687b90333cb7 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:40.238638 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0760edcc-6588-4776-a280-687b90333cb7 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:40.242367 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943d179-bd04-4d3b-b9e2-00d7b367daa9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:40.242950 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943d179-bd04-4d3b-b9e2-00d7b367daa9 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:40.243222 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943d179-bd04-4d3b-b9e2-00d7b367daa9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:40.243487 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943d179-bd04-4d3b-b9e2-00d7b367daa9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:40.245211 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6943d179-bd04-4d3b-b9e2-00d7b367daa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:40.254955 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943d179-bd04-4d3b-b9e2-00d7b367daa9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:40.256344 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6943d179-bd04-4d3b-b9e2-00d7b367daa9 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:40.264645 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1381/2782] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:40 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:49.228241 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcd5ac-af7f-4e0a-8e0a-115c9fc6ce82 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:48:49.238977 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbcd5ac-af7f-4e0a-8e0a-115c9fc6ce82 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:48:49.242545 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe2230-2ab5-43d2-bf16-62b323aac27f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:48:49.243132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe2230-2ab5-43d2-bf16-62b323aac27f service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:48:49.243427 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe2230-2ab5-43d2-bf16-62b323aac27f service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:48:49.243700 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe2230-2ab5-43d2-bf16-62b323aac27f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:48:49.245404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-aefe2230-2ab5-43d2-bf16-62b323aac27f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:48:49.256124 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe2230-2ab5-43d2-bf16-62b323aac27f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:48:49.257572 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefe2230-2ab5-43d2-bf16-62b323aac27f service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:48:49.265782 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1402/2783] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:48:49 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:48:50.260649 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=12 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:48:59.262018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=9 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:49:55.589229 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943d179-bd04-4d3b-b9e2-00d7b367daa9 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:49:55.590674 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-6943d179-bd04-4d3b-b9e2-00d7b367daa9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:49:55.605119 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6943d179-bd04-4d3b-b9e2-00d7b367daa9 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:49:55.610130 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5353db7-7959-406a-b33b-3d890556828b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:49:55.610753 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5353db7-7959-406a-b33b-3d890556828b service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:49:55.611027 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5353db7-7959-406a-b33b-3d890556828b service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:49:55.611298 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5353db7-7959-406a-b33b-3d890556828b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:49:55.619886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5353db7-7959-406a-b33b-3d890556828b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:49:55.621385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5353db7-7959-406a-b33b-3d890556828b service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:49:55.632233 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1382/2784] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:49:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:50:05.627985 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:08.554022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe2230-2ab5-43d2-bf16-62b323aac27f service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:08.566796 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe2230-2ab5-43d2-bf16-62b323aac27f service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:08.572543 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf66dac-08b5-479f-8b6b-765f252d792b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:08.573130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf66dac-08b5-479f-8b6b-765f252d792b service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:08.573403 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf66dac-08b5-479f-8b6b-765f252d792b service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:08.573697 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf66dac-08b5-479f-8b6b-765f252d792b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:08.581998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf66dac-08b5-479f-8b6b-765f252d792b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:08.583626 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf66dac-08b5-479f-8b6b-765f252d792b service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:08.593210 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1403/2785] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:50:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:50:18.588578 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:27.065853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5353db7-7959-406a-b33b-3d890556828b service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:27.075196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5353db7-7959-406a-b33b-3d890556828b service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:27.079282 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1162f-42b9-4959-a3a6-3fd7709c4430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:27.079916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1162f-42b9-4959-a3a6-3fd7709c4430 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:27.080195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1162f-42b9-4959-a3a6-3fd7709c4430 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:27.080466 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a1162f-42b9-4959-a3a6-3fd7709c4430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:27.089373 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1162f-42b9-4959-a3a6-3fd7709c4430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:27.090830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a1162f-42b9-4959-a3a6-3fd7709c4430 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:27.118694 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1383/2786] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:50:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:50:30.167018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf66dac-08b5-479f-8b6b-765f252d792b service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:30.179081 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf66dac-08b5-479f-8b6b-765f252d792b service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:30.188201 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8102f2-1fa1-4ed3-8de1-d7e241d9480e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:30.189196 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8102f2-1fa1-4ed3-8de1-d7e241d9480e service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:30.189555 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8102f2-1fa1-4ed3-8de1-d7e241d9480e service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:30.189886 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8102f2-1fa1-4ed3-8de1-d7e241d9480e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:30.191832 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-aa8102f2-1fa1-4ed3-8de1-d7e241d9480e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:30.205010 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8102f2-1fa1-4ed3-8de1-d7e241d9480e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:30.206943 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8102f2-1fa1-4ed3-8de1-d7e241d9480e service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:30.220526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1404/2787] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:50:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:50:30.226103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1162f-42b9-4959-a3a6-3fd7709c4430 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:30.238580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a1162f-42b9-4959-a3a6-3fd7709c4430 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:30.242908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e94623-bb30-4ae1-beb2-ee403a9964dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:30.244084 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e94623-bb30-4ae1-beb2-ee403a9964dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:30.244084 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e94623-bb30-4ae1-beb2-ee403a9964dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:30.244313 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e94623-bb30-4ae1-beb2-ee403a9964dd admin admin] PATH_INFO: `/v3/users/1f1ba27117d54534b9b423df00d43aa1` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:30.245317 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e94623-bb30-4ae1-beb2-ee403a9964dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f1ba27117d54534b9b423df00d43aa1)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:30.247775 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e94623-bb30-4ae1-beb2-ee403a9964dd admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:30.298005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-54e94623-bb30-4ae1-beb2-ee403a9964dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f1ba27117d54534b9b423df00d43aa1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:30.298707 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-54e94623-bb30-4ae1-beb2-ee403a9964dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f1ba27117d54534b9b423df00d43aa1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:30.310409 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-54e94623-bb30-4ae1-beb2-ee403a9964dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f1ba27117d54534b9b423df00d43aa1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:30.318988 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-54e94623-bb30-4ae1-beb2-ee403a9964dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f1ba27117d54534b9b423df00d43aa1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:30.319937 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-54e94623-bb30-4ae1-beb2-ee403a9964dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f1ba27117d54534b9b423df00d43aa1'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:30.332180 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1384/2788] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:50:30 2022] DELETE /identity/v3/users/1f1ba27117d54534b9b423df00d43aa1 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:30.505628 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8102f2-1fa1-4ed3-8de1-d7e241d9480e service placement] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:30.517220 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8102f2-1fa1-4ed3-8de1-d7e241d9480e service placement] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:30.581266 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986e4ae-330c-4a6a-bb3b-7236e97ec618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:30.581923 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e4ae-330c-4a6a-bb3b-7236e97ec618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:30.582243 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e4ae-330c-4a6a-bb3b-7236e97ec618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:30.582522 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986e4ae-330c-4a6a-bb3b-7236e97ec618 admin admin] PATH_INFO: `/v3/projects/c07806353e51499b99117ea363f78b0c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:30.583639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986e4ae-330c-4a6a-bb3b-7236e97ec618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c07806353e51499b99117ea363f78b0c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:30.585537 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986e4ae-330c-4a6a-bb3b-7236e97ec618 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:30.634546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9986e4ae-330c-4a6a-bb3b-7236e97ec618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c07806353e51499b99117ea363f78b0c', 'request_id': 'req-9986e4ae-330c-4a6a-bb3b-7236e97ec618'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:30.643036 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9986e4ae-330c-4a6a-bb3b-7236e97ec618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c07806353e51499b99117ea363f78b0c', 'request_id': 'req-9986e4ae-330c-4a6a-bb3b-7236e97ec618'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:30.651849 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9986e4ae-330c-4a6a-bb3b-7236e97ec618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c07806353e51499b99117ea363f78b0c', 'request_id': 'req-9986e4ae-330c-4a6a-bb3b-7236e97ec618'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:30.676223 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9986e4ae-330c-4a6a-bb3b-7236e97ec618 admin admin] The token cache is being invalidate because project c07806353e51499b99117ea363f78b0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81619) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:50:30.676515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9986e4ae-330c-4a6a-bb3b-7236e97ec618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:30.677172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-9986e4ae-330c-4a6a-bb3b-7236e97ec618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:30.678783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1405/2789] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:50:30 2022] DELETE /identity/v3/projects/c07806353e51499b99117ea363f78b0c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:30.683972 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2282c71-5f2a-4874-9428-b20d42e5e9c7 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:30.684258 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2282c71-5f2a-4874-9428-b20d42e5e9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:30.684527 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2282c71-5f2a-4874-9428-b20d42e5e9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:30.747853 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e2282c71-5f2a-4874-9428-b20d42e5e9c7 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:30.749073 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e2282c71-5f2a-4874-9428-b20d42e5e9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:30.811983 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1385/2790] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:30.820546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591148e6-7d8f-4a1f-88d0-b2398a3b40a9 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:30.820833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591148e6-7d8f-4a1f-88d0-b2398a3b40a9 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:30.821103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591148e6-7d8f-4a1f-88d0-b2398a3b40a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:30.840431 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-591148e6-7d8f-4a1f-88d0-b2398a3b40a9 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:30.842556 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-591148e6-7d8f-4a1f-88d0-b2398a3b40a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:30.850096 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1406/2791] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:30.854296 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2282c71-5f2a-4874-9428-b20d42e5e9c7 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:30.863838 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2282c71-5f2a-4874-9428-b20d42e5e9c7 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:30.868358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b92c87-56f9-4cf0-a0c3-70295f267372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:30.868952 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b92c87-56f9-4cf0-a0c3-70295f267372 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:30.869226 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b92c87-56f9-4cf0-a0c3-70295f267372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:30.869495 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b92c87-56f9-4cf0-a0c3-70295f267372 admin admin] PATH_INFO: `/v3/domains` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:30.870133 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b92c87-56f9-4cf0-a0c3-70295f267372 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81620) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:50:30.870462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b92c87-56f9-4cf0-a0c3-70295f267372 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:30.872036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b92c87-56f9-4cf0-a0c3-70295f267372 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:30.880487 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:30.883966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1386/2792] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:50:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:50:30.889376 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591148e6-7d8f-4a1f-88d0-b2398a3b40a9 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:30.899075 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591148e6-7d8f-4a1f-88d0-b2398a3b40a9 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:30.905095 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce577e-e430-4af1-8a4a-477663427430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:30.905629 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce577e-e430-4af1-8a4a-477663427430 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:30.905947 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce577e-e430-4af1-8a4a-477663427430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:30.906195 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce577e-e430-4af1-8a4a-477663427430 admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:30.906867 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce577e-e430-4af1-8a4a-477663427430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:30.908819 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce577e-e430-4af1-8a4a-477663427430 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:30.944669 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1407/2793] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:50:30 2022] POST /identity/v3/projects => generated 363 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:50:30.948925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b92c87-56f9-4cf0-a0c3-70295f267372 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:30.959408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b92c87-56f9-4cf0-a0c3-70295f267372 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:31.016068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dde97-9d0b-41a7-8fde-61e6a409f9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:31.016677 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dde97-9d0b-41a7-8fde-61e6a409f9ac admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:31.016957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dde97-9d0b-41a7-8fde-61e6a409f9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:31.017230 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188dde97-9d0b-41a7-8fde-61e6a409f9ac admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:31.017849 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188dde97-9d0b-41a7-8fde-61e6a409f9ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:31.019777 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188dde97-9d0b-41a7-8fde-61e6a409f9ac admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:31.040173 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1387/2794] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:50:30 2022] POST /identity/v3/users => generated 282 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:50:31.044365 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce577e-e430-4af1-8a4a-477663427430 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:31.054856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce577e-e430-4af1-8a4a-477663427430 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:31.060031 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250fcc9-a412-4491-8e0d-46e34dc18fe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:31.061208 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250fcc9-a412-4491-8e0d-46e34dc18fe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:31.061490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250fcc9-a412-4491-8e0d-46e34dc18fe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:31.061761 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250fcc9-a412-4491-8e0d-46e34dc18fe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:31.062496 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250fcc9-a412-4491-8e0d-46e34dc18fe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:31.064109 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250fcc9-a412-4491-8e0d-46e34dc18fe4 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:31.078904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1408/2795] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:50:31.080111 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dde97-9d0b-41a7-8fde-61e6a409f9ac admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:31.090111 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188dde97-9d0b-41a7-8fde-61e6a409f9ac admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:31.094860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ae08c-d048-42fc-aa02-9fe25ea1b0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:31.095467 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ae08c-d048-42fc-aa02-9fe25ea1b0e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:31.095834 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ae08c-d048-42fc-aa02-9fe25ea1b0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:31.096615 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ae08c-d048-42fc-aa02-9fe25ea1b0e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:31.096827 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ae08c-d048-42fc-aa02-9fe25ea1b0e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:31.098440 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196ae08c-d048-42fc-aa02-9fe25ea1b0e2 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:31.109374 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1388/2796] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:50:31.114777 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250fcc9-a412-4491-8e0d-46e34dc18fe4 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:31.125547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250fcc9-a412-4491-8e0d-46e34dc18fe4 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:31.130209 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd11b5-59be-44c2-8ac3-267ed8a92bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:31.130904 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd11b5-59be-44c2-8ac3-267ed8a92bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:31.131159 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd11b5-59be-44c2-8ac3-267ed8a92bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:31.131429 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cd11b5-59be-44c2-8ac3-267ed8a92bbe admin admin] PATH_INFO: `/v3/projects/52d3b2b81eac4558bad85b9ca21e6bcd/users/9d18820d9aa44ec8b79cd574677715dc/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:31.146772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd11b5-59be-44c2-8ac3-267ed8a92bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=52d3b2b81eac4558bad85b9ca21e6bcd, user_id=9d18820d9aa44ec8b79cd574677715dc, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:31.149430 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cd11b5-59be-44c2-8ac3-267ed8a92bbe admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:31.151783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:31.164926 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1409/2797] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:50:31 2022] PUT /identity/v3/projects/52d3b2b81eac4558bad85b9ca21e6bcd/users/9d18820d9aa44ec8b79cd574677715dc/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:31.227997 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bff682-e0e6-4b93-a923-9c3b6e59fd7c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:31.228560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bff682-e0e6-4b93-a923-9c3b6e59fd7c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:31.228560 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bff682-e0e6-4b93-a923-9c3b6e59fd7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:31.256917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d0bff682-e0e6-4b93-a923-9c3b6e59fd7c None None] MFA Rules not processed for user `9d18820d9aa44ec8b79cd574677715dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:31.258136 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d0bff682-e0e6-4b93-a923-9c3b6e59fd7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:31.320675 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1389/2798] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:31.325135 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd11b5-59be-44c2-8ac3-267ed8a92bbe admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:31.389774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cd11b5-59be-44c2-8ac3-267ed8a92bbe admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:31.394594 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc55146-c30c-42d0-aa55-97155768de0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:31.395186 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc55146-c30c-42d0-aa55-97155768de0c admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:31.395460 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc55146-c30c-42d0-aa55-97155768de0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:31.395724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc55146-c30c-42d0-aa55-97155768de0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:31.396373 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc55146-c30c-42d0-aa55-97155768de0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:31.398272 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc55146-c30c-42d0-aa55-97155768de0c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:31.425738 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1410/2799] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:50:31 2022] POST /identity/v3/projects => generated 365 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:50:31.430029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bff682-e0e6-4b93-a923-9c3b6e59fd7c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:31.441443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bff682-e0e6-4b93-a923-9c3b6e59fd7c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:31.497259 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a294a91-bff8-410f-ac3c-80b56226e2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:31.498491 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a294a91-bff8-410f-ac3c-80b56226e2c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:31.498768 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a294a91-bff8-410f-ac3c-80b56226e2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:31.499040 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a294a91-bff8-410f-ac3c-80b56226e2c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:31.499668 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a294a91-bff8-410f-ac3c-80b56226e2c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:31.501587 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a294a91-bff8-410f-ac3c-80b56226e2c6 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:31.517348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:31.528352 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1390/2800] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:50:31 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:50:31.530119 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc55146-c30c-42d0-aa55-97155768de0c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:31.540291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc55146-c30c-42d0-aa55-97155768de0c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:31.545735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdb00b-accc-4244-b019-0f224ec2c633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:31.546367 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdb00b-accc-4244-b019-0f224ec2c633 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:31.546645 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdb00b-accc-4244-b019-0f224ec2c633 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:31.546913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cdb00b-accc-4244-b019-0f224ec2c633 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:31.547552 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cdb00b-accc-4244-b019-0f224ec2c633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:31.549137 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cdb00b-accc-4244-b019-0f224ec2c633 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:31.558984 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1411/2801] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:50:31.563795 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a294a91-bff8-410f-ac3c-80b56226e2c6 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:31.573750 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a294a91-bff8-410f-ac3c-80b56226e2c6 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:31.578203 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f706ce-f7c4-4e74-9701-612216a07c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:31.578872 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f706ce-f7c4-4e74-9701-612216a07c8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:31.579150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f706ce-f7c4-4e74-9701-612216a07c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:31.579420 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f706ce-f7c4-4e74-9701-612216a07c8b admin admin] PATH_INFO: `/v3/projects/89d7874376834ed6ba5156dc3676c939/users/ae7cfdf3e8f14d32b67cd82214c7852d/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:31.593695 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f706ce-f7c4-4e74-9701-612216a07c8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=89d7874376834ed6ba5156dc3676c939, user_id=ae7cfdf3e8f14d32b67cd82214c7852d, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:31.596379 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f706ce-f7c4-4e74-9701-612216a07c8b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:31.609957 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1391/2802] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:50:31 2022] PUT /identity/v3/projects/89d7874376834ed6ba5156dc3676c939/users/ae7cfdf3e8f14d32b67cd82214c7852d/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:50:31.613893 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdb00b-accc-4244-b019-0f224ec2c633 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:31.624634 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cdb00b-accc-4244-b019-0f224ec2c633 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:31.680442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1702578b-db2b-4f8e-b6f1-df074e5abfbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:31.681050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702578b-db2b-4f8e-b6f1-df074e5abfbd admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:31.681327 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702578b-db2b-4f8e-b6f1-df074e5abfbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:31.681599 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1702578b-db2b-4f8e-b6f1-df074e5abfbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:31.682244 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702578b-db2b-4f8e-b6f1-df074e5abfbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:31.683835 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1702578b-db2b-4f8e-b6f1-df074e5abfbd admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:31.693940 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1412/2803] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:50:31 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:50:31.698272 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f706ce-f7c4-4e74-9701-612216a07c8b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:31.708525 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f706ce-f7c4-4e74-9701-612216a07c8b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:31.713075 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3850e-32e7-42ee-ad8c-a042a09ef7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:31.713719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3850e-32e7-42ee-ad8c-a042a09ef7cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:31.714030 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3850e-32e7-42ee-ad8c-a042a09ef7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:31.714331 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e3850e-32e7-42ee-ad8c-a042a09ef7cf admin admin] PATH_INFO: `/v3/projects/89d7874376834ed6ba5156dc3676c939/users/ae7cfdf3e8f14d32b67cd82214c7852d/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:31.716241 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3850e-32e7-42ee-ad8c-a042a09ef7cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=89d7874376834ed6ba5156dc3676c939, user_id=ae7cfdf3e8f14d32b67cd82214c7852d, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:31.719367 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e3850e-32e7-42ee-ad8c-a042a09ef7cf admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:31.729918 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1392/2804] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:50:31 2022] PUT /identity/v3/projects/89d7874376834ed6ba5156dc3676c939/users/ae7cfdf3e8f14d32b67cd82214c7852d/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:31.740205 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a137d-f509-4175-aed0-4082a4ebd769 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:31.740486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a137d-f509-4175-aed0-4082a4ebd769 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:31.740759 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a137d-f509-4175-aed0-4082a4ebd769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:31.767501 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-530a137d-f509-4175-aed0-4082a4ebd769 None None] MFA Rules not processed for user `ae7cfdf3e8f14d32b67cd82214c7852d`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:31.768688 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-530a137d-f509-4175-aed0-4082a4ebd769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:31.816991 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:31.835730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1413/2805] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:31.847273 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3850e-32e7-42ee-ad8c-a042a09ef7cf admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:31.848956 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-81e3850e-32e7-42ee-ad8c-a042a09ef7cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:31.927886 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e3850e-32e7-42ee-ad8c-a042a09ef7cf admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:31.931713 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b060fe6-2c5b-4a59-ab79-2bdab60b8966 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:31.932629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b060fe6-2c5b-4a59-ab79-2bdab60b8966 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:31.932629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b060fe6-2c5b-4a59-ab79-2bdab60b8966 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:31.932874 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b060fe6-2c5b-4a59-ab79-2bdab60b8966 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:31.941743 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b060fe6-2c5b-4a59-ab79-2bdab60b8966 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:31.943191 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b060fe6-2c5b-4a59-ab79-2bdab60b8966 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:31.952402 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1393/2806] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:50:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:50:32.366457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a137d-f509-4175-aed0-4082a4ebd769 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:32.428119 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a137d-f509-4175-aed0-4082a4ebd769 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:32.432662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511312eb-b8ca-4959-a860-5dc0dab32370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:32.433286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511312eb-b8ca-4959-a860-5dc0dab32370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:32.433545 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511312eb-b8ca-4959-a860-5dc0dab32370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:32.433809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511312eb-b8ca-4959-a860-5dc0dab32370 admin admin] PATH_INFO: `/v3/users/9d18820d9aa44ec8b79cd574677715dc` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:32.434917 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511312eb-b8ca-4959-a860-5dc0dab32370 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d18820d9aa44ec8b79cd574677715dc)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:32.436863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511312eb-b8ca-4959-a860-5dc0dab32370 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:32.482823 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-511312eb-b8ca-4959-a860-5dc0dab32370 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d18820d9aa44ec8b79cd574677715dc'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:32.490010 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-511312eb-b8ca-4959-a860-5dc0dab32370 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d18820d9aa44ec8b79cd574677715dc'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:32.502468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-511312eb-b8ca-4959-a860-5dc0dab32370 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d18820d9aa44ec8b79cd574677715dc'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:32.503147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-511312eb-b8ca-4959-a860-5dc0dab32370 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d18820d9aa44ec8b79cd574677715dc'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:32.510052 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-511312eb-b8ca-4959-a860-5dc0dab32370 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d18820d9aa44ec8b79cd574677715dc'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:32.516425 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1414/2807] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:50:32 2022] DELETE /identity/v3/users/9d18820d9aa44ec8b79cd574677715dc => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:32.520904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc328c8-1682-4259-b2d9-33c64ccb66b2 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:32.521389 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc328c8-1682-4259-b2d9-33c64ccb66b2 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:32.521514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc328c8-1682-4259-b2d9-33c64ccb66b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:32.539835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-fdc328c8-1682-4259-b2d9-33c64ccb66b2 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:32.541053 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fdc328c8-1682-4259-b2d9-33c64ccb66b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:32.598196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1394/2808] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:32.608012 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511312eb-b8ca-4959-a860-5dc0dab32370 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:32.609433 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-511312eb-b8ca-4959-a860-5dc0dab32370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:32.666568 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511312eb-b8ca-4959-a860-5dc0dab32370 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:32.670429 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7cfee-d6c6-43a2-a3c9-c29594488eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:32.671018 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7cfee-d6c6-43a2-a3c9-c29594488eed service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:32.671291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7cfee-d6c6-43a2-a3c9-c29594488eed service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:32.671557 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f7cfee-d6c6-43a2-a3c9-c29594488eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:32.680442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7cfee-d6c6-43a2-a3c9-c29594488eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:32.681856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f7cfee-d6c6-43a2-a3c9-c29594488eed service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:32.691775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1415/2809] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:50:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:50:32.964701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc328c8-1682-4259-b2d9-33c64ccb66b2 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:32.966120 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-fdc328c8-1682-4259-b2d9-33c64ccb66b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:32.978206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc328c8-1682-4259-b2d9-33c64ccb66b2 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:32.984111 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a8901-22f8-48dc-a6fb-494b4b3f4634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:32.984763 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a8901-22f8-48dc-a6fb-494b4b3f4634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:32.985068 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a8901-22f8-48dc-a6fb-494b4b3f4634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:32.985337 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a8901-22f8-48dc-a6fb-494b4b3f4634 admin admin] PATH_INFO: `/v3/projects/52d3b2b81eac4558bad85b9ca21e6bcd` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:32.986510 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a8901-22f8-48dc-a6fb-494b4b3f4634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52d3b2b81eac4558bad85b9ca21e6bcd)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:32.988430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92a8901-22f8-48dc-a6fb-494b4b3f4634 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:33.031944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:33.040104 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b92a8901-22f8-48dc-a6fb-494b4b3f4634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52d3b2b81eac4558bad85b9ca21e6bcd', 'request_id': 'req-b92a8901-22f8-48dc-a6fb-494b4b3f4634'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:33.045974 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b92a8901-22f8-48dc-a6fb-494b4b3f4634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52d3b2b81eac4558bad85b9ca21e6bcd', 'request_id': 'req-b92a8901-22f8-48dc-a6fb-494b4b3f4634'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:33.053699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b92a8901-22f8-48dc-a6fb-494b4b3f4634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52d3b2b81eac4558bad85b9ca21e6bcd', 'request_id': 'req-b92a8901-22f8-48dc-a6fb-494b4b3f4634'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:33.065753 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b92a8901-22f8-48dc-a6fb-494b4b3f4634 admin admin] The token cache is being invalidate because project 52d3b2b81eac4558bad85b9ca21e6bcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:50:33.066237 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b92a8901-22f8-48dc-a6fb-494b4b3f4634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:33.067186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-b92a8901-22f8-48dc-a6fb-494b4b3f4634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:33.068897 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1395/2810] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:50:32 2022] DELETE /identity/v3/projects/52d3b2b81eac4558bad85b9ca21e6bcd => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:33.073124 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7cfee-d6c6-43a2-a3c9-c29594488eed service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:33.084226 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f7cfee-d6c6-43a2-a3c9-c29594488eed service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:33.091891 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:33.141803 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ef5570-de20-44f3-8dca-a7d5bffa1228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:33.142449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef5570-de20-44f3-8dca-a7d5bffa1228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:33.142727 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef5570-de20-44f3-8dca-a7d5bffa1228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:33.142997 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ef5570-de20-44f3-8dca-a7d5bffa1228 admin admin] PATH_INFO: `/v3/users/ae7cfdf3e8f14d32b67cd82214c7852d` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:33.144099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef5570-de20-44f3-8dca-a7d5bffa1228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae7cfdf3e8f14d32b67cd82214c7852d)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:33.146042 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ef5570-de20-44f3-8dca-a7d5bffa1228 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:33.193136 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-21ef5570-de20-44f3-8dca-a7d5bffa1228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae7cfdf3e8f14d32b67cd82214c7852d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:33.202764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-21ef5570-de20-44f3-8dca-a7d5bffa1228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae7cfdf3e8f14d32b67cd82214c7852d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:33.215415 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-21ef5570-de20-44f3-8dca-a7d5bffa1228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae7cfdf3e8f14d32b67cd82214c7852d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:33.216003 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-21ef5570-de20-44f3-8dca-a7d5bffa1228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae7cfdf3e8f14d32b67cd82214c7852d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:33.223288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-21ef5570-de20-44f3-8dca-a7d5bffa1228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae7cfdf3e8f14d32b67cd82214c7852d'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:33.229011 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1416/2811] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:50:33 2022] DELETE /identity/v3/users/ae7cfdf3e8f14d32b67cd82214c7852d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:33.496143 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a8901-22f8-48dc-a6fb-494b4b3f4634 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:33.497599 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b92a8901-22f8-48dc-a6fb-494b4b3f4634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:33.511383 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a8901-22f8-48dc-a6fb-494b4b3f4634 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:33.568659 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58122af6-c1b8-4adc-b88f-8f2a8adb046e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:33.569323 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58122af6-c1b8-4adc-b88f-8f2a8adb046e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:33.569547 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58122af6-c1b8-4adc-b88f-8f2a8adb046e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:33.569817 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58122af6-c1b8-4adc-b88f-8f2a8adb046e admin admin] PATH_INFO: `/v3/projects/89d7874376834ed6ba5156dc3676c939` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:33.571007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58122af6-c1b8-4adc-b88f-8f2a8adb046e admin admin] RBAC: Authorizing `identity:delete_project(project_id=89d7874376834ed6ba5156dc3676c939)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:33.572890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58122af6-c1b8-4adc-b88f-8f2a8adb046e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:33.618418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-58122af6-c1b8-4adc-b88f-8f2a8adb046e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89d7874376834ed6ba5156dc3676c939', 'request_id': 'req-58122af6-c1b8-4adc-b88f-8f2a8adb046e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:33.623802 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-58122af6-c1b8-4adc-b88f-8f2a8adb046e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89d7874376834ed6ba5156dc3676c939', 'request_id': 'req-58122af6-c1b8-4adc-b88f-8f2a8adb046e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:33.630579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-58122af6-c1b8-4adc-b88f-8f2a8adb046e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89d7874376834ed6ba5156dc3676c939', 'request_id': 'req-58122af6-c1b8-4adc-b88f-8f2a8adb046e'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:33.641986 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-58122af6-c1b8-4adc-b88f-8f2a8adb046e admin admin] The token cache is being invalidate because project 89d7874376834ed6ba5156dc3676c939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:50:33.642301 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-58122af6-c1b8-4adc-b88f-8f2a8adb046e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:33.642920 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-58122af6-c1b8-4adc-b88f-8f2a8adb046e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:33.644394 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1396/2812] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:50:33 2022] DELETE /identity/v3/projects/89d7874376834ed6ba5156dc3676c939 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:33.649285 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0e532-eed5-45c1-a2dc-7b0720db623c None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:33.649792 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0e532-eed5-45c1-a2dc-7b0720db623c None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:33.649910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0e532-eed5-45c1-a2dc-7b0720db623c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:33.670058 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-b0c0e532-eed5-45c1-a2dc-7b0720db623c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:33.671293 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-b0c0e532-eed5-45c1-a2dc-7b0720db623c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:33.732178 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1417/2813] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:33.741376 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2b195-2524-4e14-8149-b1432ef5b069 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:33.741595 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2b195-2524-4e14-8149-b1432ef5b069 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:33.741865 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad2b195-2524-4e14-8149-b1432ef5b069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:33.761067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2ad2b195-2524-4e14-8149-b1432ef5b069 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:33.762276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2ad2b195-2524-4e14-8149-b1432ef5b069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:33.770468 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1397/2814] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:33.774899 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0e532-eed5-45c1-a2dc-7b0720db623c None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:33.784809 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0e532-eed5-45c1-a2dc-7b0720db623c None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:33.789330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26592a41-d583-4a3b-97ab-d8d89f61fd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:33.789914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26592a41-d583-4a3b-97ab-d8d89f61fd92 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:33.790188 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26592a41-d583-4a3b-97ab-d8d89f61fd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:33.790489 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26592a41-d583-4a3b-97ab-d8d89f61fd92 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:33.791122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26592a41-d583-4a3b-97ab-d8d89f61fd92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:50:33.791417 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26592a41-d583-4a3b-97ab-d8d89f61fd92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:33.793000 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26592a41-d583-4a3b-97ab-d8d89f61fd92 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:33.804749 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1418/2815] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:50:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:50:33.809007 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2b195-2524-4e14-8149-b1432ef5b069 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:33.818488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad2b195-2524-4e14-8149-b1432ef5b069 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:33.823066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65d95d-1475-463d-b219-3aefcd34439d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:33.823652 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65d95d-1475-463d-b219-3aefcd34439d admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:33.823930 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65d95d-1475-463d-b219-3aefcd34439d admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:33.824226 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa65d95d-1475-463d-b219-3aefcd34439d admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:33.824849 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65d95d-1475-463d-b219-3aefcd34439d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:33.827360 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa65d95d-1475-463d-b219-3aefcd34439d admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:33.854925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1398/2816] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:50:33 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:50:33.859185 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26592a41-d583-4a3b-97ab-d8d89f61fd92 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:33.869262 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26592a41-d583-4a3b-97ab-d8d89f61fd92 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:33.881277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:33.926732 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b29a3-5d02-4be7-98ac-64fe03c39b19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:33.927328 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b29a3-5d02-4be7-98ac-64fe03c39b19 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:33.927608 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b29a3-5d02-4be7-98ac-64fe03c39b19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:33.927891 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b29a3-5d02-4be7-98ac-64fe03c39b19 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:33.928501 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b29a3-5d02-4be7-98ac-64fe03c39b19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:33.930436 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888b29a3-5d02-4be7-98ac-64fe03c39b19 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:33.950504 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1419/2817] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:50:33 2022] POST /identity/v3/users => generated 282 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:50:33.955108 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65d95d-1475-463d-b219-3aefcd34439d admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:33.965329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa65d95d-1475-463d-b219-3aefcd34439d admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:33.970430 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ee182-b2d6-4e56-a518-fe8e90ce66a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:33.971186 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ee182-b2d6-4e56-a518-fe8e90ce66a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:33.971461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ee182-b2d6-4e56-a518-fe8e90ce66a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:33.971723 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ee182-b2d6-4e56-a518-fe8e90ce66a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:33.972348 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ee182-b2d6-4e56-a518-fe8e90ce66a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:33.974003 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ee182-b2d6-4e56-a518-fe8e90ce66a1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:33.983730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1399/2818] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:50:33.987575 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b29a3-5d02-4be7-98ac-64fe03c39b19 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:33.998168 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b29a3-5d02-4be7-98ac-64fe03c39b19 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:34.002902 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa2978f-3027-4955-aa1e-eaed708e661c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:34.003490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2978f-3027-4955-aa1e-eaed708e661c admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:34.003767 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2978f-3027-4955-aa1e-eaed708e661c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:34.004037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2978f-3027-4955-aa1e-eaed708e661c admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:34.004672 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2978f-3027-4955-aa1e-eaed708e661c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:34.006277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa2978f-3027-4955-aa1e-eaed708e661c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:34.018764 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1420/2819] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:50:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:50:34.022924 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ee182-b2d6-4e56-a518-fe8e90ce66a1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:34.032806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ee182-b2d6-4e56-a518-fe8e90ce66a1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:34.037279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33732cc-82d7-46ba-be6e-a0078538cc3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:34.037999 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33732cc-82d7-46ba-be6e-a0078538cc3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:34.038307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33732cc-82d7-46ba-be6e-a0078538cc3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:34.038575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33732cc-82d7-46ba-be6e-a0078538cc3c admin admin] PATH_INFO: `/v3/projects/9c1f5af7a2384d8c8026193b34d45568/users/68e398f4880f4a588057ac703564f3c9/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:34.054328 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33732cc-82d7-46ba-be6e-a0078538cc3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c1f5af7a2384d8c8026193b34d45568, user_id=68e398f4880f4a588057ac703564f3c9, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:34.056937 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33732cc-82d7-46ba-be6e-a0078538cc3c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:34.059191 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:34.071081 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1400/2820] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:50:34 2022] PUT /identity/v3/projects/9c1f5af7a2384d8c8026193b34d45568/users/68e398f4880f4a588057ac703564f3c9/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:34.082007 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043477b4-94dc-48f3-bdcc-605cefdeeaa4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:34.082338 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043477b4-94dc-48f3-bdcc-605cefdeeaa4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:34.082640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043477b4-94dc-48f3-bdcc-605cefdeeaa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:34.110241 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-043477b4-94dc-48f3-bdcc-605cefdeeaa4 None None] MFA Rules not processed for user `68e398f4880f4a588057ac703564f3c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:34.111488 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-043477b4-94dc-48f3-bdcc-605cefdeeaa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:34.174982 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1421/2821] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:34.179568 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33732cc-82d7-46ba-be6e-a0078538cc3c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:34.240309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33732cc-82d7-46ba-be6e-a0078538cc3c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:34.245062 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b3968-7413-499b-aabf-4f1e6163199f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:34.245655 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b3968-7413-499b-aabf-4f1e6163199f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:34.245929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b3968-7413-499b-aabf-4f1e6163199f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:34.246196 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b3968-7413-499b-aabf-4f1e6163199f admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:34.246843 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b3968-7413-499b-aabf-4f1e6163199f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:34.249281 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b3968-7413-499b-aabf-4f1e6163199f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:34.275045 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1401/2822] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:50:34 2022] POST /identity/v3/projects => generated 365 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:50:34.279167 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043477b4-94dc-48f3-bdcc-605cefdeeaa4 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:34.289859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043477b4-94dc-48f3-bdcc-605cefdeeaa4 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:34.345740 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5039722-808e-4375-a795-bbc91f85e676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:34.346369 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5039722-808e-4375-a795-bbc91f85e676 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:34.346648 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5039722-808e-4375-a795-bbc91f85e676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:34.346919 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5039722-808e-4375-a795-bbc91f85e676 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:34.347532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5039722-808e-4375-a795-bbc91f85e676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:34.349466 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5039722-808e-4375-a795-bbc91f85e676 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:34.370065 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1422/2823] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:50:34 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:50:34.374731 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b3968-7413-499b-aabf-4f1e6163199f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:34.385429 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b3968-7413-499b-aabf-4f1e6163199f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:34.390067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00270d0-dafb-4e72-8718-97835b815aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:34.390708 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00270d0-dafb-4e72-8718-97835b815aba admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:34.391027 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00270d0-dafb-4e72-8718-97835b815aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:34.391297 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00270d0-dafb-4e72-8718-97835b815aba admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:34.391923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00270d0-dafb-4e72-8718-97835b815aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:34.393482 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00270d0-dafb-4e72-8718-97835b815aba admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:34.404090 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1402/2824] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:50:34.408326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5039722-808e-4375-a795-bbc91f85e676 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:34.418336 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5039722-808e-4375-a795-bbc91f85e676 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:34.423370 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31dbf4-6837-4c10-85ff-f3a2c9b00548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:34.424015 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31dbf4-6837-4c10-85ff-f3a2c9b00548 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:34.424273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31dbf4-6837-4c10-85ff-f3a2c9b00548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:34.424538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d31dbf4-6837-4c10-85ff-f3a2c9b00548 admin admin] PATH_INFO: `/v3/projects/aa8b3cc48d684dbea948f878e23096c5/users/67d08d3ee9b94b1f95a938d2afe3b093/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:34.436750 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:34.440386 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31dbf4-6837-4c10-85ff-f3a2c9b00548 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8b3cc48d684dbea948f878e23096c5, user_id=67d08d3ee9b94b1f95a938d2afe3b093, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:34.443338 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d31dbf4-6837-4c10-85ff-f3a2c9b00548 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:34.457529 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1423/2825] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:50:34 2022] PUT /identity/v3/projects/aa8b3cc48d684dbea948f878e23096c5/users/67d08d3ee9b94b1f95a938d2afe3b093/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:34.461454 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00270d0-dafb-4e72-8718-97835b815aba admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:34.471396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00270d0-dafb-4e72-8718-97835b815aba admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:34.531000 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5397d8f-d524-40aa-b423-d0b7f4cffc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:34.531814 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5397d8f-d524-40aa-b423-d0b7f4cffc4e admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:34.532176 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5397d8f-d524-40aa-b423-d0b7f4cffc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:34.532488 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5397d8f-d524-40aa-b423-d0b7f4cffc4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:34.533187 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5397d8f-d524-40aa-b423-d0b7f4cffc4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:34.534909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5397d8f-d524-40aa-b423-d0b7f4cffc4e admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:34.545589 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1403/2826] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:50:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:50:34.550085 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31dbf4-6837-4c10-85ff-f3a2c9b00548 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:34.561105 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d31dbf4-6837-4c10-85ff-f3a2c9b00548 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:34.565708 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2101c-de7b-49e1-9540-1d07cb741758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:34.566377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2101c-de7b-49e1-9540-1d07cb741758 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:34.566652 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2101c-de7b-49e1-9540-1d07cb741758 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:34.566919 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb2101c-de7b-49e1-9540-1d07cb741758 admin admin] PATH_INFO: `/v3/projects/aa8b3cc48d684dbea948f878e23096c5/users/67d08d3ee9b94b1f95a938d2afe3b093/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:34.568789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2101c-de7b-49e1-9540-1d07cb741758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa8b3cc48d684dbea948f878e23096c5, user_id=67d08d3ee9b94b1f95a938d2afe3b093, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:34.571425 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb2101c-de7b-49e1-9540-1d07cb741758 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:34.581163 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1424/2827] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:50:34 2022] PUT /identity/v3/projects/aa8b3cc48d684dbea948f878e23096c5/users/67d08d3ee9b94b1f95a938d2afe3b093/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:34.591252 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a864a-7a3b-43c1-afdc-693ae3abb937 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:34.591703 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a864a-7a3b-43c1-afdc-693ae3abb937 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:34.591815 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1a864a-7a3b-43c1-afdc-693ae3abb937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:34.618904 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-9a1a864a-7a3b-43c1-afdc-693ae3abb937 None None] MFA Rules not processed for user `67d08d3ee9b94b1f95a938d2afe3b093`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:34.620114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9a1a864a-7a3b-43c1-afdc-693ae3abb937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:34.668401 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:34.688583 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1404/2828] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:34 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:34.699894 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2101c-de7b-49e1-9540-1d07cb741758 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:34.701611 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-bbb2101c-de7b-49e1-9540-1d07cb741758 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:34.760007 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb2101c-de7b-49e1-9540-1d07cb741758 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:34.763793 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d3643-a3e8-4139-b867-57ab3f1e04c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:34.764392 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d3643-a3e8-4139-b867-57ab3f1e04c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:34.764673 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d3643-a3e8-4139-b867-57ab3f1e04c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:34.764958 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5d3643-a3e8-4139-b867-57ab3f1e04c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:34.773710 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d3643-a3e8-4139-b867-57ab3f1e04c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:34.775175 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5d3643-a3e8-4139-b867-57ab3f1e04c3 service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:34.785064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1425/2829] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:50:34 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:50:34.861225 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a864a-7a3b-43c1-afdc-693ae3abb937 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:34.873067 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1a864a-7a3b-43c1-afdc-693ae3abb937 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:34.877169 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08f166-53a6-44f9-853a-02ca6a99d1a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:34.877833 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08f166-53a6-44f9-853a-02ca6a99d1a7 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:34.878114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08f166-53a6-44f9-853a-02ca6a99d1a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:34.878408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08f166-53a6-44f9-853a-02ca6a99d1a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:34.935215 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08f166-53a6-44f9-853a-02ca6a99d1a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:34.936673 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08f166-53a6-44f9-853a-02ca6a99d1a7 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:34.945355 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1405/2830] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:50:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:50:44.780856 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=34,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:44.941476 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=33 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:46.209748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d3643-a3e8-4139-b867-57ab3f1e04c3 service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:46.271319 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5d3643-a3e8-4139-b867-57ab3f1e04c3 service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:46.275869 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6b378-fe38-4d2f-8bb2-09910c0f0b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:46.276488 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6b378-fe38-4d2f-8bb2-09910c0f0b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:46.276773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6b378-fe38-4d2f-8bb2-09910c0f0b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:46.277046 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6b378-fe38-4d2f-8bb2-09910c0f0b8a admin admin] PATH_INFO: `/v3/users/68e398f4880f4a588057ac703564f3c9` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:46.278153 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6b378-fe38-4d2f-8bb2-09910c0f0b8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=68e398f4880f4a588057ac703564f3c9)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:46.280159 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6b378-fe38-4d2f-8bb2-09910c0f0b8a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:46.326010 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cdc6b378-fe38-4d2f-8bb2-09910c0f0b8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68e398f4880f4a588057ac703564f3c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:46.332970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cdc6b378-fe38-4d2f-8bb2-09910c0f0b8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68e398f4880f4a588057ac703564f3c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:46.344655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cdc6b378-fe38-4d2f-8bb2-09910c0f0b8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68e398f4880f4a588057ac703564f3c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:46.345273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cdc6b378-fe38-4d2f-8bb2-09910c0f0b8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68e398f4880f4a588057ac703564f3c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:46.352059 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-cdc6b378-fe38-4d2f-8bb2-09910c0f0b8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68e398f4880f4a588057ac703564f3c9'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:46.358377 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1426/2831] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:50:46 2022] DELETE /identity/v3/users/68e398f4880f4a588057ac703564f3c9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:46.362922 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d34bd4-a684-48a4-b87a-60abbadea9f8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:46.363394 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d34bd4-a684-48a4-b87a-60abbadea9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:46.363508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d34bd4-a684-48a4-b87a-60abbadea9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:46.382956 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-c6d34bd4-a684-48a4-b87a-60abbadea9f8 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:46.384168 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c6d34bd4-a684-48a4-b87a-60abbadea9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:46.444307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1406/2832] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:46.454130 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6b378-fe38-4d2f-8bb2-09910c0f0b8a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:46.455570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cdc6b378-fe38-4d2f-8bb2-09910c0f0b8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:46.513149 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6b378-fe38-4d2f-8bb2-09910c0f0b8a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:46.516836 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96782437-60af-4ba9-8169-1088075396e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:46.517442 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96782437-60af-4ba9-8169-1088075396e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:46.517721 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96782437-60af-4ba9-8169-1088075396e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:46.517995 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96782437-60af-4ba9-8169-1088075396e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:46.526963 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96782437-60af-4ba9-8169-1088075396e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:46.528409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96782437-60af-4ba9-8169-1088075396e0 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:46.538083 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1427/2833] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:50:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:50:46.807005 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d34bd4-a684-48a4-b87a-60abbadea9f8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:46.808458 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-c6d34bd4-a684-48a4-b87a-60abbadea9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:46.822155 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d34bd4-a684-48a4-b87a-60abbadea9f8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:46.827052 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877b682-45cd-4b54-ac0c-c3dd787744cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:46.827681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877b682-45cd-4b54-ac0c-c3dd787744cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:46.827962 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877b682-45cd-4b54-ac0c-c3dd787744cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:46.828230 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877b682-45cd-4b54-ac0c-c3dd787744cc admin admin] PATH_INFO: `/v3/projects/9c1f5af7a2384d8c8026193b34d45568` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:46.829359 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877b682-45cd-4b54-ac0c-c3dd787744cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c1f5af7a2384d8c8026193b34d45568)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:46.831313 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9877b682-45cd-4b54-ac0c-c3dd787744cc admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:46.877358 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9877b682-45cd-4b54-ac0c-c3dd787744cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c1f5af7a2384d8c8026193b34d45568', 'request_id': 'req-9877b682-45cd-4b54-ac0c-c3dd787744cc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:46.883119 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9877b682-45cd-4b54-ac0c-c3dd787744cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c1f5af7a2384d8c8026193b34d45568', 'request_id': 'req-9877b682-45cd-4b54-ac0c-c3dd787744cc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:46.890419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9877b682-45cd-4b54-ac0c-c3dd787744cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c1f5af7a2384d8c8026193b34d45568', 'request_id': 'req-9877b682-45cd-4b54-ac0c-c3dd787744cc'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:46.901496 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9877b682-45cd-4b54-ac0c-c3dd787744cc admin admin] The token cache is being invalidate because project 9c1f5af7a2384d8c8026193b34d45568 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:50:46.901784 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9877b682-45cd-4b54-ac0c-c3dd787744cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:46.902472 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-9877b682-45cd-4b54-ac0c-c3dd787744cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:46.904436 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1407/2834] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:50:46 2022] DELETE /identity/v3/projects/9c1f5af7a2384d8c8026193b34d45568 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:46.908569 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96782437-60af-4ba9-8169-1088075396e0 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:46.918981 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96782437-60af-4ba9-8169-1088075396e0 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:46.937479 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:46.976531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa5ea53-c075-4ce8-b41d-134d72984dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:46.977150 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5ea53-c075-4ce8-b41d-134d72984dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:46.977426 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5ea53-c075-4ce8-b41d-134d72984dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:46.977695 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa5ea53-c075-4ce8-b41d-134d72984dad admin admin] PATH_INFO: `/v3/users/67d08d3ee9b94b1f95a938d2afe3b093` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:46.978833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5ea53-c075-4ce8-b41d-134d72984dad admin admin] RBAC: Authorizing `identity:delete_user(user_id=67d08d3ee9b94b1f95a938d2afe3b093)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:46.980735 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa5ea53-c075-4ce8-b41d-134d72984dad admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:47.027527 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dfa5ea53-c075-4ce8-b41d-134d72984dad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67d08d3ee9b94b1f95a938d2afe3b093'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:47.035536 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dfa5ea53-c075-4ce8-b41d-134d72984dad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67d08d3ee9b94b1f95a938d2afe3b093'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:47.048434 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dfa5ea53-c075-4ce8-b41d-134d72984dad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67d08d3ee9b94b1f95a938d2afe3b093'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:47.049070 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dfa5ea53-c075-4ce8-b41d-134d72984dad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67d08d3ee9b94b1f95a938d2afe3b093'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:47.055881 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-dfa5ea53-c075-4ce8-b41d-134d72984dad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67d08d3ee9b94b1f95a938d2afe3b093'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:47.061789 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1428/2835] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:50:46 2022] DELETE /identity/v3/users/67d08d3ee9b94b1f95a938d2afe3b093 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:47.332251 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877b682-45cd-4b54-ac0c-c3dd787744cc admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:47.333852 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-9877b682-45cd-4b54-ac0c-c3dd787744cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:47.347635 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877b682-45cd-4b54-ac0c-c3dd787744cc admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:47.404525 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aebe80-0a70-4e3d-8ea5-b66ac7a15191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:47.405243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aebe80-0a70-4e3d-8ea5-b66ac7a15191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:47.405425 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aebe80-0a70-4e3d-8ea5-b66ac7a15191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:47.405698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aebe80-0a70-4e3d-8ea5-b66ac7a15191 admin admin] PATH_INFO: `/v3/projects/aa8b3cc48d684dbea948f878e23096c5` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:47.406837 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aebe80-0a70-4e3d-8ea5-b66ac7a15191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa8b3cc48d684dbea948f878e23096c5)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:47.408741 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aebe80-0a70-4e3d-8ea5-b66ac7a15191 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:47.440511 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:47.459235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41aebe80-0a70-4e3d-8ea5-b66ac7a15191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa8b3cc48d684dbea948f878e23096c5', 'request_id': 'req-41aebe80-0a70-4e3d-8ea5-b66ac7a15191'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:47.464847 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41aebe80-0a70-4e3d-8ea5-b66ac7a15191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa8b3cc48d684dbea948f878e23096c5', 'request_id': 'req-41aebe80-0a70-4e3d-8ea5-b66ac7a15191'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:47.472868 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41aebe80-0a70-4e3d-8ea5-b66ac7a15191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa8b3cc48d684dbea948f878e23096c5', 'request_id': 'req-41aebe80-0a70-4e3d-8ea5-b66ac7a15191'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:47.483944 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41aebe80-0a70-4e3d-8ea5-b66ac7a15191 admin admin] The token cache is being invalidate because project aa8b3cc48d684dbea948f878e23096c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:50:47.484235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41aebe80-0a70-4e3d-8ea5-b66ac7a15191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:47.484879 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-41aebe80-0a70-4e3d-8ea5-b66ac7a15191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:47.486390 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1408/2836] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:50:47 2022] DELETE /identity/v3/projects/aa8b3cc48d684dbea948f878e23096c5 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:47.491329 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20864a78-cde6-4f74-b27d-23e6de42c80a None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:47.491821 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20864a78-cde6-4f74-b27d-23e6de42c80a None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:47.491946 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20864a78-cde6-4f74-b27d-23e6de42c80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:47.513644 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-20864a78-cde6-4f74-b27d-23e6de42c80a None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:47.515050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-20864a78-cde6-4f74-b27d-23e6de42c80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:47.577166 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1429/2837] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:47.585163 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1f3cc-06f1-4cb7-bc67-5ffc780341f8 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:47.585452 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1f3cc-06f1-4cb7-bc67-5ffc780341f8 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:47.585723 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1f3cc-06f1-4cb7-bc67-5ffc780341f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:47.605031 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-dca1f3cc-06f1-4cb7-bc67-5ffc780341f8 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:47.606261 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-dca1f3cc-06f1-4cb7-bc67-5ffc780341f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:47.614276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1409/2838] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:47.618401 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20864a78-cde6-4f74-b27d-23e6de42c80a None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:47.628483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20864a78-cde6-4f74-b27d-23e6de42c80a None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:47.632964 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d676b38-e97e-4c9e-aba0-d60bb256f10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:47.633531 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d676b38-e97e-4c9e-aba0-d60bb256f10d admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:47.633804 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d676b38-e97e-4c9e-aba0-d60bb256f10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:47.634066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d676b38-e97e-4c9e-aba0-d60bb256f10d admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:47.634730 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d676b38-e97e-4c9e-aba0-d60bb256f10d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:50:47.635027 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d676b38-e97e-4c9e-aba0-d60bb256f10d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:47.636591 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d676b38-e97e-4c9e-aba0-d60bb256f10d admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:47.648062 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1430/2839] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:50:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:50:47.652235 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1f3cc-06f1-4cb7-bc67-5ffc780341f8 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:47.661387 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1f3cc-06f1-4cb7-bc67-5ffc780341f8 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:47.666009 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb2abd-4a4a-4821-b3b7-625479545a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:47.666639 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb2abd-4a4a-4821-b3b7-625479545a2f admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:47.666916 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb2abd-4a4a-4821-b3b7-625479545a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:47.667182 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cb2abd-4a4a-4821-b3b7-625479545a2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:47.667806 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb2abd-4a4a-4821-b3b7-625479545a2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:47.669941 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cb2abd-4a4a-4821-b3b7-625479545a2f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:47.696598 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1410/2840] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:50:47 2022] POST /identity/v3/projects => generated 409 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:50:47.701592 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d676b38-e97e-4c9e-aba0-d60bb256f10d admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:47.711441 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d676b38-e97e-4c9e-aba0-d60bb256f10d admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:47.736969 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:47.769423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcac038-521a-4941-99b9-3a6aac674538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:47.770019 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcac038-521a-4941-99b9-3a6aac674538 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:47.770330 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcac038-521a-4941-99b9-3a6aac674538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:47.770606 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcac038-521a-4941-99b9-3a6aac674538 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:47.771238 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcac038-521a-4941-99b9-3a6aac674538 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:47.773147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcac038-521a-4941-99b9-3a6aac674538 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:47.793696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1431/2841] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:50:47 2022] POST /identity/v3/users => generated 305 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:50:47.798118 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb2abd-4a4a-4821-b3b7-625479545a2f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:47.808278 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cb2abd-4a4a-4821-b3b7-625479545a2f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:47.813393 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003885d5-15d8-49e6-b4bd-78ce6b6277a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:47.813982 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003885d5-15d8-49e6-b4bd-78ce6b6277a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:47.814288 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003885d5-15d8-49e6-b4bd-78ce6b6277a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:47.814562 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003885d5-15d8-49e6-b4bd-78ce6b6277a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:47.815222 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003885d5-15d8-49e6-b4bd-78ce6b6277a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:47.817069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003885d5-15d8-49e6-b4bd-78ce6b6277a5 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:47.827671 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1411/2842] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:50:47.831549 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcac038-521a-4941-99b9-3a6aac674538 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:47.841465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcac038-521a-4941-99b9-3a6aac674538 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:47.846127 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f0914d-a00e-4ae6-ba74-0f686d6d9ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:47.846747 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0914d-a00e-4ae6-ba74-0f686d6d9ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:47.847023 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0914d-a00e-4ae6-ba74-0f686d6d9ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:47.847291 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0914d-a00e-4ae6-ba74-0f686d6d9ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:47.847922 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0914d-a00e-4ae6-ba74-0f686d6d9ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:47.849526 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f0914d-a00e-4ae6-ba74-0f686d6d9ed0 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:47.861163 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1432/2843] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:50:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:50:47.865314 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003885d5-15d8-49e6-b4bd-78ce6b6277a5 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:47.875707 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003885d5-15d8-49e6-b4bd-78ce6b6277a5 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:47.880293 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c42ec5-9325-46bf-a591-487ed44467df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:47.880926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c42ec5-9325-46bf-a591-487ed44467df admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:47.881197 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c42ec5-9325-46bf-a591-487ed44467df admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:47.881462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c42ec5-9325-46bf-a591-487ed44467df admin admin] PATH_INFO: `/v3/projects/7322a59b96324cdb902de9cca7263ce8/users/79551c2f8443469096ed6a6018e3e045/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:47.895705 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c42ec5-9325-46bf-a591-487ed44467df admin admin] RBAC: Authorizing `identity:create_grant(project_id=7322a59b96324cdb902de9cca7263ce8, user_id=79551c2f8443469096ed6a6018e3e045, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:47.898329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c42ec5-9325-46bf-a591-487ed44467df admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:47.911218 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1412/2844] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:50:47 2022] PUT /identity/v3/projects/7322a59b96324cdb902de9cca7263ce8/users/79551c2f8443469096ed6a6018e3e045/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:47.922326 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43066e-4867-45f1-9908-021f5f55be34 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:47.922611 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43066e-4867-45f1-9908-021f5f55be34 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:47.922925 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee43066e-4867-45f1-9908-021f5f55be34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:47.950952 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-ee43066e-4867-45f1-9908-021f5f55be34 None None] MFA Rules not processed for user `79551c2f8443469096ed6a6018e3e045`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:47.952155 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-ee43066e-4867-45f1-9908-021f5f55be34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:48.015137 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1433/2845] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:47 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:48.019636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c42ec5-9325-46bf-a591-487ed44467df admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:48.080189 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c42ec5-9325-46bf-a591-487ed44467df admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:48.084835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b9552-ba45-4d5f-9d93-cea1d5d519e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:48.085431 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b9552-ba45-4d5f-9d93-cea1d5d519e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:48.085707 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b9552-ba45-4d5f-9d93-cea1d5d519e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:48.085975 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b9552-ba45-4d5f-9d93-cea1d5d519e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:48.086718 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b9552-ba45-4d5f-9d93-cea1d5d519e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:48.088628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124b9552-ba45-4d5f-9d93-cea1d5d519e1 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:48.115131 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1413/2846] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:50:48 2022] POST /identity/v3/projects => generated 407 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:50:48.119211 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43066e-4867-45f1-9908-021f5f55be34 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:48.130322 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee43066e-4867-45f1-9908-021f5f55be34 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:48.185875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52313812-2d3a-4271-b0ef-5e2494d3adcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:48.186422 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52313812-2d3a-4271-b0ef-5e2494d3adcc admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:48.186697 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52313812-2d3a-4271-b0ef-5e2494d3adcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:48.186962 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52313812-2d3a-4271-b0ef-5e2494d3adcc admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:48.187570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52313812-2d3a-4271-b0ef-5e2494d3adcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:48.189458 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52313812-2d3a-4271-b0ef-5e2494d3adcc admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:48.211277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1434/2847] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:50:48 2022] POST /identity/v3/users => generated 310 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:50:48.215550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b9552-ba45-4d5f-9d93-cea1d5d519e1 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:48.226441 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124b9552-ba45-4d5f-9d93-cea1d5d519e1 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:48.230975 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb71887-28b9-4352-a7f6-bf02aa9d16c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:48.231616 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb71887-28b9-4352-a7f6-bf02aa9d16c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:48.232416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb71887-28b9-4352-a7f6-bf02aa9d16c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:48.232682 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb71887-28b9-4352-a7f6-bf02aa9d16c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:48.233305 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb71887-28b9-4352-a7f6-bf02aa9d16c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:48.234926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb71887-28b9-4352-a7f6-bf02aa9d16c0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:48.245305 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1414/2848] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:50:48.249441 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52313812-2d3a-4271-b0ef-5e2494d3adcc admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:48.259660 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52313812-2d3a-4271-b0ef-5e2494d3adcc admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:48.264268 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec59f64-6dba-4d0b-a0f4-cdef20963779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:48.264908 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec59f64-6dba-4d0b-a0f4-cdef20963779 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:48.265189 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec59f64-6dba-4d0b-a0f4-cdef20963779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:48.265462 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec59f64-6dba-4d0b-a0f4-cdef20963779 admin admin] PATH_INFO: `/v3/projects/ceea7777e9da49e4bc555bcc2b4d14fb/users/4b91e4b66ada45738e4c718e331d3a52/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:48.280449 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec59f64-6dba-4d0b-a0f4-cdef20963779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceea7777e9da49e4bc555bcc2b4d14fb, user_id=4b91e4b66ada45738e4c718e331d3a52, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:48.283138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec59f64-6dba-4d0b-a0f4-cdef20963779 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:48.297151 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1435/2849] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:50:48 2022] PUT /identity/v3/projects/ceea7777e9da49e4bc555bcc2b4d14fb/users/4b91e4b66ada45738e4c718e331d3a52/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:48.301059 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb71887-28b9-4352-a7f6-bf02aa9d16c0 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:48.310871 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb71887-28b9-4352-a7f6-bf02aa9d16c0 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:48.330027 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:48.367210 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19618c9-d655-4a48-a16b-b1e6a5531088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:48.367805 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19618c9-d655-4a48-a16b-b1e6a5531088 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:48.368078 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19618c9-d655-4a48-a16b-b1e6a5531088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:48.368344 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19618c9-d655-4a48-a16b-b1e6a5531088 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:48.368964 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19618c9-d655-4a48-a16b-b1e6a5531088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:48.370564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19618c9-d655-4a48-a16b-b1e6a5531088 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:48.380128 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1415/2850] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:50:48.384852 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec59f64-6dba-4d0b-a0f4-cdef20963779 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:48.390331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:48.396536 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec59f64-6dba-4d0b-a0f4-cdef20963779 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:48.401931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff836e-edbb-4132-acba-607d653d534e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:48.402628 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff836e-edbb-4132-acba-607d653d534e admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:48.402907 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff836e-edbb-4132-acba-607d653d534e admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:48.403177 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ff836e-edbb-4132-acba-607d653d534e admin admin] PATH_INFO: `/v3/projects/ceea7777e9da49e4bc555bcc2b4d14fb/users/4b91e4b66ada45738e4c718e331d3a52/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:48.405587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff836e-edbb-4132-acba-607d653d534e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ceea7777e9da49e4bc555bcc2b4d14fb, user_id=4b91e4b66ada45738e4c718e331d3a52, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:48.408333 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ff836e-edbb-4132-acba-607d653d534e admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:48.418523 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1436/2851] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:50:48 2022] PUT /identity/v3/projects/ceea7777e9da49e4bc555bcc2b4d14fb/users/4b91e4b66ada45738e4c718e331d3a52/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:48.430764 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da1008-0f38-4983-a807-17152a3cf64c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:48.431267 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da1008-0f38-4983-a807-17152a3cf64c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:48.431462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da1008-0f38-4983-a807-17152a3cf64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:48.458730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-62da1008-0f38-4983-a807-17152a3cf64c None None] MFA Rules not processed for user `4b91e4b66ada45738e4c718e331d3a52`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:48.459549 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-62da1008-0f38-4983-a807-17152a3cf64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:48.525915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1416/2852] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:48 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:48.537159 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff836e-edbb-4132-acba-607d653d534e admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:48.538931 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-48ff836e-edbb-4132-acba-607d653d534e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:48.597365 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ff836e-edbb-4132-acba-607d653d534e admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:48.600949 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987a348-cb39-4fb3-aabd-9edec8d529df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:48.601538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987a348-cb39-4fb3-aabd-9edec8d529df service cinder] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:48.601813 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987a348-cb39-4fb3-aabd-9edec8d529df service cinder] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:48.602122 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987a348-cb39-4fb3-aabd-9edec8d529df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:48.611653 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987a348-cb39-4fb3-aabd-9edec8d529df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:48.613076 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987a348-cb39-4fb3-aabd-9edec8d529df service cinder] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:48.622532 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1437/2853] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:50:48 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:50:48.769658 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da1008-0f38-4983-a807-17152a3cf64c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:48.779846 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62da1008-0f38-4983-a807-17152a3cf64c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:48.783372 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be166f66-effd-46e1-8e8d-498418f4a071 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:48.783971 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be166f66-effd-46e1-8e8d-498418f4a071 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:48.784243 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be166f66-effd-46e1-8e8d-498418f4a071 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:48.784510 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be166f66-effd-46e1-8e8d-498418f4a071 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:48.840462 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be166f66-effd-46e1-8e8d-498418f4a071 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:48.842479 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be166f66-effd-46e1-8e8d-498418f4a071 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:48.851172 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1417/2854] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:50:48 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:50:57.172160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987a348-cb39-4fb3-aabd-9edec8d529df service cinder] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:57.235137 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987a348-cb39-4fb3-aabd-9edec8d529df service cinder] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:57.239773 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ea3dc-1611-4eab-b4d8-ae22caaa5072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:57.240389 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ea3dc-1611-4eab-b4d8-ae22caaa5072 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:57.240665 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ea3dc-1611-4eab-b4d8-ae22caaa5072 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:57.240935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ea3dc-1611-4eab-b4d8-ae22caaa5072 admin admin] PATH_INFO: `/v3/users/79551c2f8443469096ed6a6018e3e045` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:57.242050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ea3dc-1611-4eab-b4d8-ae22caaa5072 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79551c2f8443469096ed6a6018e3e045)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:57.243979 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ea3dc-1611-4eab-b4d8-ae22caaa5072 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:57.291873 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2c6ea3dc-1611-4eab-b4d8-ae22caaa5072 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79551c2f8443469096ed6a6018e3e045'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:57.299596 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2c6ea3dc-1611-4eab-b4d8-ae22caaa5072 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79551c2f8443469096ed6a6018e3e045'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:57.312110 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2c6ea3dc-1611-4eab-b4d8-ae22caaa5072 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79551c2f8443469096ed6a6018e3e045'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:57.312738 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2c6ea3dc-1611-4eab-b4d8-ae22caaa5072 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79551c2f8443469096ed6a6018e3e045'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:57.319505 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2c6ea3dc-1611-4eab-b4d8-ae22caaa5072 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79551c2f8443469096ed6a6018e3e045'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:57.325616 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1438/2855] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:50:57 2022] DELETE /identity/v3/users/79551c2f8443469096ed6a6018e3e045 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:57.330239 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63af79c-ccda-4114-8a4d-6f36887d8154 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:57.330771 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63af79c-ccda-4114-8a4d-6f36887d8154 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:57.330891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63af79c-ccda-4114-8a4d-6f36887d8154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:57.351023 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-d63af79c-ccda-4114-8a4d-6f36887d8154 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:57.352280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d63af79c-ccda-4114-8a4d-6f36887d8154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:57.390887 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=100 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:57.409540 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1418/2856] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:57.420080 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ea3dc-1611-4eab-b4d8-ae22caaa5072 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:57.421483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2c6ea3dc-1611-4eab-b4d8-ae22caaa5072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:57.434724 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:57.479701 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ea3dc-1611-4eab-b4d8-ae22caaa5072 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:57.483922 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b6229-3028-40d3-ab0e-50449c600345 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:57.484513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b6229-3028-40d3-ab0e-50449c600345 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:57.484788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b6229-3028-40d3-ab0e-50449c600345 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:57.485051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b6229-3028-40d3-ab0e-50449c600345 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:57.493916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b6229-3028-40d3-ab0e-50449c600345 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:57.495363 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b6229-3028-40d3-ab0e-50449c600345 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:57.505255 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1439/2857] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:50:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:50:58.025692 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63af79c-ccda-4114-8a4d-6f36887d8154 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:58.027652 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-d63af79c-ccda-4114-8a4d-6f36887d8154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:58.040735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63af79c-ccda-4114-8a4d-6f36887d8154 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:58.046148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d594231-d13f-45f1-90bb-401eedf85f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:58.046816 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d594231-d13f-45f1-90bb-401eedf85f1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:58.047092 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d594231-d13f-45f1-90bb-401eedf85f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:58.047365 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d594231-d13f-45f1-90bb-401eedf85f1c admin admin] PATH_INFO: `/v3/projects/7322a59b96324cdb902de9cca7263ce8` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:58.048471 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d594231-d13f-45f1-90bb-401eedf85f1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7322a59b96324cdb902de9cca7263ce8)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:58.050619 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d594231-d13f-45f1-90bb-401eedf85f1c admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:58.098201 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d594231-d13f-45f1-90bb-401eedf85f1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7322a59b96324cdb902de9cca7263ce8', 'request_id': 'req-7d594231-d13f-45f1-90bb-401eedf85f1c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:58.103860 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d594231-d13f-45f1-90bb-401eedf85f1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7322a59b96324cdb902de9cca7263ce8', 'request_id': 'req-7d594231-d13f-45f1-90bb-401eedf85f1c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:58.113249 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d594231-d13f-45f1-90bb-401eedf85f1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7322a59b96324cdb902de9cca7263ce8', 'request_id': 'req-7d594231-d13f-45f1-90bb-401eedf85f1c'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:58.125095 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d594231-d13f-45f1-90bb-401eedf85f1c admin admin] The token cache is being invalidate because project 7322a59b96324cdb902de9cca7263ce8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:50:58.125385 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d594231-d13f-45f1-90bb-401eedf85f1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:58.126011 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-7d594231-d13f-45f1-90bb-401eedf85f1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:58.127508 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1419/2858] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:50:58 2022] DELETE /identity/v3/projects/7322a59b96324cdb902de9cca7263ce8 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:58.131570 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b6229-3028-40d3-ab0e-50449c600345 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:58.141181 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b6229-3028-40d3-ab0e-50449c600345 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:58.196396 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c91daf0-432f-4c5f-8265-f48891e8f506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:58.197099 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91daf0-432f-4c5f-8265-f48891e8f506 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:58.197277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91daf0-432f-4c5f-8265-f48891e8f506 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:58.197542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c91daf0-432f-4c5f-8265-f48891e8f506 admin admin] PATH_INFO: `/v3/users/4b91e4b66ada45738e4c718e331d3a52` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:58.198679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91daf0-432f-4c5f-8265-f48891e8f506 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b91e4b66ada45738e4c718e331d3a52)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:58.200584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c91daf0-432f-4c5f-8265-f48891e8f506 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:58.246279 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3c91daf0-432f-4c5f-8265-f48891e8f506 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b91e4b66ada45738e4c718e331d3a52'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:58.254959 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3c91daf0-432f-4c5f-8265-f48891e8f506 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b91e4b66ada45738e4c718e331d3a52'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:58.266872 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3c91daf0-432f-4c5f-8265-f48891e8f506 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b91e4b66ada45738e4c718e331d3a52'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:58.267505 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3c91daf0-432f-4c5f-8265-f48891e8f506 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b91e4b66ada45738e4c718e331d3a52'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:58.274658 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-3c91daf0-432f-4c5f-8265-f48891e8f506 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b91e4b66ada45738e4c718e331d3a52'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:58.281411 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1440/2859] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:50:58 2022] DELETE /identity/v3/users/4b91e4b66ada45738e4c718e331d3a52 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:58.550183 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d594231-d13f-45f1-90bb-401eedf85f1c admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:58.551680 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7d594231-d13f-45f1-90bb-401eedf85f1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:58.564822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d594231-d13f-45f1-90bb-401eedf85f1c admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:58.622396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6663d8d1-6e44-416d-8461-83cb689d5d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:58.623029 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663d8d1-6e44-416d-8461-83cb689d5d48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:58.623299 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663d8d1-6e44-416d-8461-83cb689d5d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:58.623564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663d8d1-6e44-416d-8461-83cb689d5d48 admin admin] PATH_INFO: `/v3/projects/ceea7777e9da49e4bc555bcc2b4d14fb` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:58.624666 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663d8d1-6e44-416d-8461-83cb689d5d48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ceea7777e9da49e4bc555bcc2b4d14fb)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:58.626606 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6663d8d1-6e44-416d-8461-83cb689d5d48 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:58.672526 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6663d8d1-6e44-416d-8461-83cb689d5d48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ceea7777e9da49e4bc555bcc2b4d14fb', 'request_id': 'req-6663d8d1-6e44-416d-8461-83cb689d5d48'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:58.678001 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6663d8d1-6e44-416d-8461-83cb689d5d48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ceea7777e9da49e4bc555bcc2b4d14fb', 'request_id': 'req-6663d8d1-6e44-416d-8461-83cb689d5d48'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:58.688736 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6663d8d1-6e44-416d-8461-83cb689d5d48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ceea7777e9da49e4bc555bcc2b4d14fb', 'request_id': 'req-6663d8d1-6e44-416d-8461-83cb689d5d48'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:58.696019 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:58.701552 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6663d8d1-6e44-416d-8461-83cb689d5d48 admin admin] The token cache is being invalidate because project ceea7777e9da49e4bc555bcc2b4d14fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:50:58.701966 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6663d8d1-6e44-416d-8461-83cb689d5d48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:58.702993 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-6663d8d1-6e44-416d-8461-83cb689d5d48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:50:58.704554 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1420/2860] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:50:58 2022] DELETE /identity/v3/projects/ceea7777e9da49e4bc555bcc2b4d14fb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:58.711352 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156717b8-3b6d-4c96-8f01-7e28318d69e4 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:58.711635 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156717b8-3b6d-4c96-8f01-7e28318d69e4 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:58.711907 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156717b8-3b6d-4c96-8f01-7e28318d69e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:58.731486 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-156717b8-3b6d-4c96-8f01-7e28318d69e4 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:58.732624 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-156717b8-3b6d-4c96-8f01-7e28318d69e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:58.762271 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:50:58.793613 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1441/2861] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:58.801891 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ca93f-aa55-402f-a6e0-df60fd1e325c None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:58.802281 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ca93f-aa55-402f-a6e0-df60fd1e325c None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:58.802644 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936ca93f-aa55-402f-a6e0-df60fd1e325c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:58.823150 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-936ca93f-aa55-402f-a6e0-df60fd1e325c None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:58.824411 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-936ca93f-aa55-402f-a6e0-df60fd1e325c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:58.833443 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1421/2862] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:50:58.837679 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156717b8-3b6d-4c96-8f01-7e28318d69e4 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:58.847312 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156717b8-3b6d-4c96-8f01-7e28318d69e4 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:58.851998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca995965-d8ef-4a88-815d-2f08b583f706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:58.852597 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca995965-d8ef-4a88-815d-2f08b583f706 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:58.852875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca995965-d8ef-4a88-815d-2f08b583f706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:58.853145 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca995965-d8ef-4a88-815d-2f08b583f706 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:58.853791 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca995965-d8ef-4a88-815d-2f08b583f706 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:50:58.854115 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca995965-d8ef-4a88-815d-2f08b583f706 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:58.855772 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca995965-d8ef-4a88-815d-2f08b583f706 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:58.867230 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1442/2863] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:50:58.872307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ca93f-aa55-402f-a6e0-df60fd1e325c None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:58.881433 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936ca93f-aa55-402f-a6e0-df60fd1e325c None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:58.886036 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1e1b9-03fe-4269-96cb-2179d011584a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:58.886659 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1e1b9-03fe-4269-96cb-2179d011584a admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:58.886937 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1e1b9-03fe-4269-96cb-2179d011584a admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:58.887204 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf1e1b9-03fe-4269-96cb-2179d011584a admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:58.888363 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1e1b9-03fe-4269-96cb-2179d011584a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:58.890284 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf1e1b9-03fe-4269-96cb-2179d011584a admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:58.916698 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1422/2864] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:50:58 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:50:58.921103 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca995965-d8ef-4a88-815d-2f08b583f706 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:58.931178 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca995965-d8ef-4a88-815d-2f08b583f706 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:58.991288 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2584e5a-d4f6-4b02-a175-c487b32da4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:58.991827 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2584e5a-d4f6-4b02-a175-c487b32da4bf admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:58.992106 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2584e5a-d4f6-4b02-a175-c487b32da4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:58.992378 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2584e5a-d4f6-4b02-a175-c487b32da4bf admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:58.992995 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2584e5a-d4f6-4b02-a175-c487b32da4bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:58.994935 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2584e5a-d4f6-4b02-a175-c487b32da4bf admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:59.016405 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1443/2865] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:50:58 2022] POST /identity/v3/users => generated 282 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:50:59.020689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1e1b9-03fe-4269-96cb-2179d011584a admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:59.031171 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf1e1b9-03fe-4269-96cb-2179d011584a admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:59.035716 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04f227-12a7-4011-a8d1-5980ade0725f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:59.036306 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04f227-12a7-4011-a8d1-5980ade0725f admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:59.036579 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04f227-12a7-4011-a8d1-5980ade0725f admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:59.036844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f04f227-12a7-4011-a8d1-5980ade0725f admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:59.037477 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04f227-12a7-4011-a8d1-5980ade0725f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:59.039071 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f04f227-12a7-4011-a8d1-5980ade0725f admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:59.048963 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1423/2866] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:50:59.053007 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2584e5a-d4f6-4b02-a175-c487b32da4bf admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:59.063538 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2584e5a-d4f6-4b02-a175-c487b32da4bf admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:59.068402 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171121b5-0c74-437a-a370-aff347c4003b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:59.069167 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171121b5-0c74-437a-a370-aff347c4003b admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:59.069465 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171121b5-0c74-437a-a370-aff347c4003b admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:59.069774 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171121b5-0c74-437a-a370-aff347c4003b admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:59.070492 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171121b5-0c74-437a-a370-aff347c4003b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:59.072105 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171121b5-0c74-437a-a370-aff347c4003b admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:59.083457 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1444/2867] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:50:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:50:59.087835 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04f227-12a7-4011-a8d1-5980ade0725f admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:59.097915 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f04f227-12a7-4011-a8d1-5980ade0725f admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:59.102492 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248892f6-102a-4925-a239-4282fcd4c708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:59.103130 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248892f6-102a-4925-a239-4282fcd4c708 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:59.103405 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248892f6-102a-4925-a239-4282fcd4c708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:59.103672 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248892f6-102a-4925-a239-4282fcd4c708 admin admin] PATH_INFO: `/v3/projects/a58adc1e665140baac764d0d988907e7/users/752ae5bf8e5d45aebf01852dec9d1704/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:59.118850 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248892f6-102a-4925-a239-4282fcd4c708 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a58adc1e665140baac764d0d988907e7, user_id=752ae5bf8e5d45aebf01852dec9d1704, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:59.121513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248892f6-102a-4925-a239-4282fcd4c708 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:59.134643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1424/2868] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:50:59 2022] PUT /identity/v3/projects/a58adc1e665140baac764d0d988907e7/users/752ae5bf8e5d45aebf01852dec9d1704/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:50:59.145513 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f00cbe-db96-4fcd-91cc-edb6d5dda86f None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:59.145795 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f00cbe-db96-4fcd-91cc-edb6d5dda86f None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:59.146064 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f00cbe-db96-4fcd-91cc-edb6d5dda86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:59.174637 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-28f00cbe-db96-4fcd-91cc-edb6d5dda86f None None] MFA Rules not processed for user `752ae5bf8e5d45aebf01852dec9d1704`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:50:59.175877 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-28f00cbe-db96-4fcd-91cc-edb6d5dda86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:59.237142 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1445/2869] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:50:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:50:59.249395 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248892f6-102a-4925-a239-4282fcd4c708 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:50:59.251121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-248892f6-102a-4925-a239-4282fcd4c708 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:50:59.306994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248892f6-102a-4925-a239-4282fcd4c708 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:50:59.311176 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5613e0d9-7596-49d9-9540-849bf74924bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:50:59.311782 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613e0d9-7596-49d9-9540-849bf74924bf service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:50:59.312060 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613e0d9-7596-49d9-9540-849bf74924bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:50:59.312329 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5613e0d9-7596-49d9-9540-849bf74924bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:50:59.320472 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613e0d9-7596-49d9-9540-849bf74924bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:50:59.321893 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5613e0d9-7596-49d9-9540-849bf74924bf service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:50:59.330620 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1425/2870] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:50:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:51:09.234335 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:51:09.326758 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=54,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:51:39.560218 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f00cbe-db96-4fcd-91cc-edb6d5dda86f None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:39.622077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f00cbe-db96-4fcd-91cc-edb6d5dda86f None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:39.627314 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685148cf-3d2a-4907-8b9b-c0a9ba4f9d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:39.627937 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685148cf-3d2a-4907-8b9b-c0a9ba4f9d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:39.628216 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685148cf-3d2a-4907-8b9b-c0a9ba4f9d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:39.628487 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685148cf-3d2a-4907-8b9b-c0a9ba4f9d67 admin admin] PATH_INFO: `/v3/users/752ae5bf8e5d45aebf01852dec9d1704` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:39.629598 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685148cf-3d2a-4907-8b9b-c0a9ba4f9d67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=752ae5bf8e5d45aebf01852dec9d1704)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:39.631542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685148cf-3d2a-4907-8b9b-c0a9ba4f9d67 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:39.679095 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-685148cf-3d2a-4907-8b9b-c0a9ba4f9d67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '752ae5bf8e5d45aebf01852dec9d1704'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:39.687175 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-685148cf-3d2a-4907-8b9b-c0a9ba4f9d67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '752ae5bf8e5d45aebf01852dec9d1704'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:39.699093 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-685148cf-3d2a-4907-8b9b-c0a9ba4f9d67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '752ae5bf8e5d45aebf01852dec9d1704'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:39.699748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-685148cf-3d2a-4907-8b9b-c0a9ba4f9d67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '752ae5bf8e5d45aebf01852dec9d1704'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:39.706651 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-685148cf-3d2a-4907-8b9b-c0a9ba4f9d67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '752ae5bf8e5d45aebf01852dec9d1704'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:39.712359 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1446/2871] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:51:39 2022] DELETE /identity/v3/users/752ae5bf8e5d45aebf01852dec9d1704 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:51:39.716888 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888f4a0-3410-486a-a75c-b577e537d43b None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:39.717396 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888f4a0-3410-486a-a75c-b577e537d43b None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:39.717511 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e888f4a0-3410-486a-a75c-b577e537d43b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:39.736842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-e888f4a0-3410-486a-a75c-b577e537d43b None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:51:39.738047 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e888f4a0-3410-486a-a75c-b577e537d43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:39.796901 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1426/2872] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:51:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:51:39.806784 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685148cf-3d2a-4907-8b9b-c0a9ba4f9d67 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:39.808193 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-685148cf-3d2a-4907-8b9b-c0a9ba4f9d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:39.864277 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685148cf-3d2a-4907-8b9b-c0a9ba4f9d67 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:39.867946 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb4ac08-8e01-46f4-ac15-73a7a6fb94c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:39.868542 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4ac08-8e01-46f4-ac15-73a7a6fb94c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:39.868818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4ac08-8e01-46f4-ac15-73a7a6fb94c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:39.869088 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb4ac08-8e01-46f4-ac15-73a7a6fb94c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:39.878013 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb4ac08-8e01-46f4-ac15-73a7a6fb94c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:39.879480 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb4ac08-8e01-46f4-ac15-73a7a6fb94c8 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:39.888910 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1447/2873] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:51:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:51:40.154762 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888f4a0-3410-486a-a75c-b577e537d43b None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:40.156309 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-e888f4a0-3410-486a-a75c-b577e537d43b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:40.168425 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e888f4a0-3410-486a-a75c-b577e537d43b None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:40.173265 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d1468-e2bf-43b0-8878-639fd4e5f4f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:40.173884 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d1468-e2bf-43b0-8878-639fd4e5f4f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:40.174163 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d1468-e2bf-43b0-8878-639fd4e5f4f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:40.174463 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d1468-e2bf-43b0-8878-639fd4e5f4f9 admin admin] PATH_INFO: `/v3/projects/a58adc1e665140baac764d0d988907e7` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:40.175572 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d1468-e2bf-43b0-8878-639fd4e5f4f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a58adc1e665140baac764d0d988907e7)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:40.177505 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d1468-e2bf-43b0-8878-639fd4e5f4f9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:40.224408 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5a0d1468-e2bf-43b0-8878-639fd4e5f4f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a58adc1e665140baac764d0d988907e7', 'request_id': 'req-5a0d1468-e2bf-43b0-8878-639fd4e5f4f9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:40.229766 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5a0d1468-e2bf-43b0-8878-639fd4e5f4f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a58adc1e665140baac764d0d988907e7', 'request_id': 'req-5a0d1468-e2bf-43b0-8878-639fd4e5f4f9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:40.237020 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5a0d1468-e2bf-43b0-8878-639fd4e5f4f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a58adc1e665140baac764d0d988907e7', 'request_id': 'req-5a0d1468-e2bf-43b0-8878-639fd4e5f4f9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:40.248214 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5a0d1468-e2bf-43b0-8878-639fd4e5f4f9 admin admin] The token cache is being invalidate because project a58adc1e665140baac764d0d988907e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:51:40.248499 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5a0d1468-e2bf-43b0-8878-639fd4e5f4f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:40.249652 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-5a0d1468-e2bf-43b0-8878-639fd4e5f4f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:40.251167 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1427/2874] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:51:40 2022] DELETE /identity/v3/projects/a58adc1e665140baac764d0d988907e7 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:51:40.255973 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0904075-18cd-45dd-92d0-1b728564c378 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:40.256260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0904075-18cd-45dd-92d0-1b728564c378 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:40.256551 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0904075-18cd-45dd-92d0-1b728564c378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:40.277754 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f0904075-18cd-45dd-92d0-1b728564c378 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:51:40.278998 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f0904075-18cd-45dd-92d0-1b728564c378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:40.296823 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:51:40.340160 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1448/2875] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:51:40.347878 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff1a47-cc5d-4717-aae6-accf0f30dd30 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:40.348165 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff1a47-cc5d-4717-aae6-accf0f30dd30 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:40.348438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaff1a47-cc5d-4717-aae6-accf0f30dd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:40.368907 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-eaff1a47-cc5d-4717-aae6-accf0f30dd30 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:51:40.370114 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-eaff1a47-cc5d-4717-aae6-accf0f30dd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:40.378767 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1428/2876] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:51:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:51:40.382939 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0904075-18cd-45dd-92d0-1b728564c378 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:40.392738 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0904075-18cd-45dd-92d0-1b728564c378 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:40.397267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70c440-b2a4-403a-836b-3ffe65db42d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:40.397855 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70c440-b2a4-403a-836b-3ffe65db42d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:40.398129 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70c440-b2a4-403a-836b-3ffe65db42d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:40.398425 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70c440-b2a4-403a-836b-3ffe65db42d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:40.399063 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70c440-b2a4-403a-836b-3ffe65db42d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:51:40.399383 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70c440-b2a4-403a-836b-3ffe65db42d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:40.400961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70c440-b2a4-403a-836b-3ffe65db42d8 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:40.412073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1449/2877] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:51:40.416446 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff1a47-cc5d-4717-aae6-accf0f30dd30 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:40.427046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaff1a47-cc5d-4717-aae6-accf0f30dd30 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:40.432268 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a82dc4-1564-4a3f-b32e-76eb34c39705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:40.432909 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a82dc4-1564-4a3f-b32e-76eb34c39705 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:40.433188 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a82dc4-1564-4a3f-b32e-76eb34c39705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:40.433664 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a82dc4-1564-4a3f-b32e-76eb34c39705 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:40.434287 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a82dc4-1564-4a3f-b32e-76eb34c39705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:40.436975 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a82dc4-1564-4a3f-b32e-76eb34c39705 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:40.464585 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1429/2878] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:51:40 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:51:40.468894 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70c440-b2a4-403a-836b-3ffe65db42d8 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:40.478908 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70c440-b2a4-403a-836b-3ffe65db42d8 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:40.534924 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ebf80-9492-41a0-8eeb-91f687e1212c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:40.535530 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ebf80-9492-41a0-8eeb-91f687e1212c admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:40.535805 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ebf80-9492-41a0-8eeb-91f687e1212c admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:40.536073 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571ebf80-9492-41a0-8eeb-91f687e1212c admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:40.536697 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ebf80-9492-41a0-8eeb-91f687e1212c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:40.538643 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571ebf80-9492-41a0-8eeb-91f687e1212c admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:40.560914 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1450/2879] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:51:40 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:51:40.565761 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a82dc4-1564-4a3f-b32e-76eb34c39705 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:40.575923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a82dc4-1564-4a3f-b32e-76eb34c39705 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:40.580699 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba47c8c-0d44-4deb-a12a-76864f0362ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:40.581324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba47c8c-0d44-4deb-a12a-76864f0362ba admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:40.581599 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba47c8c-0d44-4deb-a12a-76864f0362ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:40.581864 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba47c8c-0d44-4deb-a12a-76864f0362ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:40.582556 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba47c8c-0d44-4deb-a12a-76864f0362ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:40.584139 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba47c8c-0d44-4deb-a12a-76864f0362ba admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:40.593925 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1430/2880] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:51:40.597874 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ebf80-9492-41a0-8eeb-91f687e1212c admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:40.607648 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571ebf80-9492-41a0-8eeb-91f687e1212c admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:40.612661 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428923cd-955a-46eb-8c1b-89ec343acb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:40.613238 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428923cd-955a-46eb-8c1b-89ec343acb3a admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:40.613525 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428923cd-955a-46eb-8c1b-89ec343acb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:40.613775 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428923cd-955a-46eb-8c1b-89ec343acb3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:40.614843 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428923cd-955a-46eb-8c1b-89ec343acb3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:40.616418 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428923cd-955a-46eb-8c1b-89ec343acb3a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:40.627432 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1451/2881] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:51:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:51:40.631831 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba47c8c-0d44-4deb-a12a-76864f0362ba admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:40.642350 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba47c8c-0d44-4deb-a12a-76864f0362ba admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:40.647222 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58bc7a0-0ed1-47f8-ad26-060a4b9b5094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:40.647879 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bc7a0-0ed1-47f8-ad26-060a4b9b5094 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:40.648158 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bc7a0-0ed1-47f8-ad26-060a4b9b5094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:40.648428 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58bc7a0-0ed1-47f8-ad26-060a4b9b5094 admin admin] PATH_INFO: `/v3/projects/da9a7a94074b4786bbfc5454f9fff1ee/users/91fc4bc6dada485f9def79c2f2c17e1c/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:40.664461 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58bc7a0-0ed1-47f8-ad26-060a4b9b5094 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da9a7a94074b4786bbfc5454f9fff1ee, user_id=91fc4bc6dada485f9def79c2f2c17e1c, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:40.667117 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58bc7a0-0ed1-47f8-ad26-060a4b9b5094 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:40.680302 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1431/2882] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:51:40 2022] PUT /identity/v3/projects/da9a7a94074b4786bbfc5454f9fff1ee/users/91fc4bc6dada485f9def79c2f2c17e1c/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:51:40.692375 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af512f-0c38-41cd-9140-59fd6eccf4a6 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:40.692663 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af512f-0c38-41cd-9140-59fd6eccf4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:40.692934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af512f-0c38-41cd-9140-59fd6eccf4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:40.720574 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-f1af512f-0c38-41cd-9140-59fd6eccf4a6 None None] MFA Rules not processed for user `91fc4bc6dada485f9def79c2f2c17e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:51:40.721771 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-f1af512f-0c38-41cd-9140-59fd6eccf4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:40.784453 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1452/2883] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:51:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:51:40.795870 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58bc7a0-0ed1-47f8-ad26-060a4b9b5094 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:40.797793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a58bc7a0-0ed1-47f8-ad26-060a4b9b5094 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:40.819742 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:51:40.857069 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58bc7a0-0ed1-47f8-ad26-060a4b9b5094 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:40.860730 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b45390-698e-4998-bf61-047c1a6ddf69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:40.861332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b45390-698e-4998-bf61-047c1a6ddf69 service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:40.861636 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b45390-698e-4998-bf61-047c1a6ddf69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:40.861905 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b45390-698e-4998-bf61-047c1a6ddf69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:40.870094 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b45390-698e-4998-bf61-047c1a6ddf69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:40.871550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b45390-698e-4998-bf61-047c1a6ddf69 service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:40.880975 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1432/2884] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:51:40 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:51:40.906051 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af512f-0c38-41cd-9140-59fd6eccf4a6 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:40.924845 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:51:40.969767 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af512f-0c38-41cd-9140-59fd6eccf4a6 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:40.974409 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea75b88-eacc-4b32-bc37-9e7594aed291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:40.975027 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea75b88-eacc-4b32-bc37-9e7594aed291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:40.975304 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea75b88-eacc-4b32-bc37-9e7594aed291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:40.975575 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea75b88-eacc-4b32-bc37-9e7594aed291 admin admin] PATH_INFO: `/v3/users/91fc4bc6dada485f9def79c2f2c17e1c` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:40.976733 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea75b88-eacc-4b32-bc37-9e7594aed291 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91fc4bc6dada485f9def79c2f2c17e1c)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:40.978680 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea75b88-eacc-4b32-bc37-9e7594aed291 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:41.024769 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fea75b88-eacc-4b32-bc37-9e7594aed291 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91fc4bc6dada485f9def79c2f2c17e1c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:41.032756 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fea75b88-eacc-4b32-bc37-9e7594aed291 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91fc4bc6dada485f9def79c2f2c17e1c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:41.044971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fea75b88-eacc-4b32-bc37-9e7594aed291 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91fc4bc6dada485f9def79c2f2c17e1c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:41.046404 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fea75b88-eacc-4b32-bc37-9e7594aed291 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91fc4bc6dada485f9def79c2f2c17e1c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:41.053250 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-fea75b88-eacc-4b32-bc37-9e7594aed291 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91fc4bc6dada485f9def79c2f2c17e1c'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:41.059169 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1453/2885] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:51:40 2022] DELETE /identity/v3/users/91fc4bc6dada485f9def79c2f2c17e1c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:51:41.064366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e413dc6-fe0f-4dfa-9eb0-1bcad7d43004 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:41.064654 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e413dc6-fe0f-4dfa-9eb0-1bcad7d43004 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:41.064923 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e413dc6-fe0f-4dfa-9eb0-1bcad7d43004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:41.083587 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-2e413dc6-fe0f-4dfa-9eb0-1bcad7d43004 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:51:41.084779 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2e413dc6-fe0f-4dfa-9eb0-1bcad7d43004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:41.141836 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1433/2886] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:51:41.151440 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea75b88-eacc-4b32-bc37-9e7594aed291 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:41.152863 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fea75b88-eacc-4b32-bc37-9e7594aed291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:41.209999 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea75b88-eacc-4b32-bc37-9e7594aed291 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:41.213638 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d548b0-95c3-40c6-8152-82a769554fdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:41.214260 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d548b0-95c3-40c6-8152-82a769554fdc service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:41.214546 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d548b0-95c3-40c6-8152-82a769554fdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:41.214818 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d548b0-95c3-40c6-8152-82a769554fdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:41.224042 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d548b0-95c3-40c6-8152-82a769554fdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:41.225483 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d548b0-95c3-40c6-8152-82a769554fdc service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:41.235452 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1454/2887] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:51:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:51:41.506073 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e413dc6-fe0f-4dfa-9eb0-1bcad7d43004 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:41.507596 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-2e413dc6-fe0f-4dfa-9eb0-1bcad7d43004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:41.519769 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e413dc6-fe0f-4dfa-9eb0-1bcad7d43004 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:41.524497 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028f9f33-3f5a-49d6-b1ed-ca00b2905dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:41.525209 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f9f33-3f5a-49d6-b1ed-ca00b2905dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:41.525555 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f9f33-3f5a-49d6-b1ed-ca00b2905dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:41.525845 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028f9f33-3f5a-49d6-b1ed-ca00b2905dee admin admin] PATH_INFO: `/v3/projects/da9a7a94074b4786bbfc5454f9fff1ee` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:41.527136 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f9f33-3f5a-49d6-b1ed-ca00b2905dee admin admin] RBAC: Authorizing `identity:delete_project(project_id=da9a7a94074b4786bbfc5454f9fff1ee)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:41.529070 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028f9f33-3f5a-49d6-b1ed-ca00b2905dee admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:41.576994 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-028f9f33-3f5a-49d6-b1ed-ca00b2905dee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da9a7a94074b4786bbfc5454f9fff1ee', 'request_id': 'req-028f9f33-3f5a-49d6-b1ed-ca00b2905dee'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:41.582629 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-028f9f33-3f5a-49d6-b1ed-ca00b2905dee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da9a7a94074b4786bbfc5454f9fff1ee', 'request_id': 'req-028f9f33-3f5a-49d6-b1ed-ca00b2905dee'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:41.589745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-028f9f33-3f5a-49d6-b1ed-ca00b2905dee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da9a7a94074b4786bbfc5454f9fff1ee', 'request_id': 'req-028f9f33-3f5a-49d6-b1ed-ca00b2905dee'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:41.601198 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-028f9f33-3f5a-49d6-b1ed-ca00b2905dee admin admin] The token cache is being invalidate because project da9a7a94074b4786bbfc5454f9fff1ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:51:41.601530 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-028f9f33-3f5a-49d6-b1ed-ca00b2905dee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:41.602151 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-028f9f33-3f5a-49d6-b1ed-ca00b2905dee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:51:41.603625 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1434/2888] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:51:41 2022] DELETE /identity/v3/projects/da9a7a94074b4786bbfc5454f9fff1ee => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:51:41.609190 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b074a-16ec-4dd6-9cca-d2284cf2bd90 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:41.609662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b074a-16ec-4dd6-9cca-d2284cf2bd90 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:41.609788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87b074a-16ec-4dd6-9cca-d2284cf2bd90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:41.628593 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-a87b074a-16ec-4dd6-9cca-d2284cf2bd90 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:51:41.629798 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a87b074a-16ec-4dd6-9cca-d2284cf2bd90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:41.663358 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:51:41.690523 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1455/2889] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:51:41.702794 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847f43d-8f81-4777-91d3-108625ff6353 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:41.703109 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847f43d-8f81-4777-91d3-108625ff6353 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:41.703388 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7847f43d-8f81-4777-91d3-108625ff6353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:41.723219 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-7847f43d-8f81-4777-91d3-108625ff6353 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:51:41.724437 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7847f43d-8f81-4777-91d3-108625ff6353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:41.732381 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1435/2890] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:51:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 15:51:41.736846 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b074a-16ec-4dd6-9cca-d2284cf2bd90 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:41.747339 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87b074a-16ec-4dd6-9cca-d2284cf2bd90 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:41.752002 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e155f-e181-46d0-8094-bd05a806b550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:41.752588 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e155f-e181-46d0-8094-bd05a806b550 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:41.752859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e155f-e181-46d0-8094-bd05a806b550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:41.753125 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161e155f-e181-46d0-8094-bd05a806b550 admin admin] PATH_INFO: `/v3/domains` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:41.753759 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e155f-e181-46d0-8094-bd05a806b550 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81619) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 15:51:41.754056 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e155f-e181-46d0-8094-bd05a806b550 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:41.755684 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161e155f-e181-46d0-8094-bd05a806b550 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:41.767834 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1456/2891] 10.222.0.76 () {58 vars in 1153 bytes} [Wed Aug 24 15:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 15:51:41.772206 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847f43d-8f81-4777-91d3-108625ff6353 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:41.782077 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7847f43d-8f81-4777-91d3-108625ff6353 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:41.786755 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d69428-3514-4f53-bba4-30f4c233fed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:41.787684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d69428-3514-4f53-bba4-30f4c233fed4 admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:41.787684 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d69428-3514-4f53-bba4-30f4c233fed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:41.787968 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d69428-3514-4f53-bba4-30f4c233fed4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:41.788587 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d69428-3514-4f53-bba4-30f4c233fed4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:41.790513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d69428-3514-4f53-bba4-30f4c233fed4 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:41.820287 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1436/2892] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:51:41 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:51:41.824547 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e155f-e181-46d0-8094-bd05a806b550 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:41.834967 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161e155f-e181-46d0-8094-bd05a806b550 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:41.891297 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87715a-8579-46a8-8f8d-b257a6cdbdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:41.891901 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87715a-8579-46a8-8f8d-b257a6cdbdde admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:41.892184 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87715a-8579-46a8-8f8d-b257a6cdbdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:41.892454 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87715a-8579-46a8-8f8d-b257a6cdbdde admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:41.893070 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87715a-8579-46a8-8f8d-b257a6cdbdde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:41.895022 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87715a-8579-46a8-8f8d-b257a6cdbdde admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:41.915821 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1457/2893] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:51:41 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:51:41.920378 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d69428-3514-4f53-bba4-30f4c233fed4 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:41.931016 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d69428-3514-4f53-bba4-30f4c233fed4 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:41.935635 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a7455-1ba4-4669-b56b-e1e48dd7ee72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:41.936230 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a7455-1ba4-4669-b56b-e1e48dd7ee72 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:41.936551 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a7455-1ba4-4669-b56b-e1e48dd7ee72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:41.936824 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a7455-1ba4-4669-b56b-e1e48dd7ee72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:41.937463 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a7455-1ba4-4669-b56b-e1e48dd7ee72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:41.939090 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00a7455-1ba4-4669-b56b-e1e48dd7ee72 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:41.948933 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1437/2894] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:51:41.953380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87715a-8579-46a8-8f8d-b257a6cdbdde admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:41.963292 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87715a-8579-46a8-8f8d-b257a6cdbdde admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:41.968387 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02295374-53e3-4ab9-a2b9-29cee6b52156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:41.968982 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02295374-53e3-4ab9-a2b9-29cee6b52156 admin admin] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:41.969250 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02295374-53e3-4ab9-a2b9-29cee6b52156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:41.969514 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02295374-53e3-4ab9-a2b9-29cee6b52156 admin admin] PATH_INFO: `/v3/roles` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:41.970143 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02295374-53e3-4ab9-a2b9-29cee6b52156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:41.971741 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02295374-53e3-4ab9-a2b9-29cee6b52156 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:41.983166 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1458/2895] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:51:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:51:41.987423 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a7455-1ba4-4669-b56b-e1e48dd7ee72 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:41.997332 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a7455-1ba4-4669-b56b-e1e48dd7ee72 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:42.001919 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daf5ed-830d-4466-a6fe-404321bfabd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:42.002592 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daf5ed-830d-4466-a6fe-404321bfabd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:42.002890 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daf5ed-830d-4466-a6fe-404321bfabd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:42.003143 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4daf5ed-830d-4466-a6fe-404321bfabd9 admin admin] PATH_INFO: `/v3/projects/ae3022891c914ea18ffacc414b55fb7e/users/1151eb3a305e4c18b8cc5717379bce63/roles/8d794ef8abb94809bd988c5db32c0037` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:42.005325 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:51:42.020921 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4daf5ed-830d-4466-a6fe-404321bfabd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3022891c914ea18ffacc414b55fb7e, user_id=1151eb3a305e4c18b8cc5717379bce63, role_id=8d794ef8abb94809bd988c5db32c0037)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:42.023624 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4daf5ed-830d-4466-a6fe-404321bfabd9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:42.038278 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1438/2896] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:51:41 2022] PUT /identity/v3/projects/ae3022891c914ea18ffacc414b55fb7e/users/1151eb3a305e4c18b8cc5717379bce63/roles/8d794ef8abb94809bd988c5db32c0037 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:51:42.043407 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928711d-c524-4056-90ff-b70b73b04f44 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:42.043692 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928711d-c524-4056-90ff-b70b73b04f44 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:42.043961 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8928711d-c524-4056-90ff-b70b73b04f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:42.063844 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-8928711d-c524-4056-90ff-b70b73b04f44 None None] MFA Rules not processed for user `793b87d316d544f9aecda76f798927dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:51:42.065037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-8928711d-c524-4056-90ff-b70b73b04f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:42.122390 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1459/2897] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:51:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:51:42.132627 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daf5ed-830d-4466-a6fe-404321bfabd9 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:42.134548 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-a4daf5ed-830d-4466-a6fe-404321bfabd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:42.190867 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4daf5ed-830d-4466-a6fe-404321bfabd9 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:42.194926 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2d6e3-fd96-4da7-9994-5458595954e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:42.195506 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2d6e3-fd96-4da7-9994-5458595954e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:42.195793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2d6e3-fd96-4da7-9994-5458595954e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:42.196098 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d2d6e3-fd96-4da7-9994-5458595954e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:42.205424 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d2d6e3-fd96-4da7-9994-5458595954e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:42.206830 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d2d6e3-fd96-4da7-9994-5458595954e3 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:42.216171 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1439/2898] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:51:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:51:46.007574 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab91f5-4f2a-4bff-9c23-533e0b4e8c76 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:46.008126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab91f5-4f2a-4bff-9c23-533e0b4e8c76 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:46.008126 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab91f5-4f2a-4bff-9c23-533e0b4e8c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:46.038031 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-d7ab91f5-4f2a-4bff-9c23-533e0b4e8c76 None None] MFA Rules not processed for user `1151eb3a305e4c18b8cc5717379bce63`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:51:46.039468 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d7ab91f5-4f2a-4bff-9c23-533e0b4e8c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:46.075639 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:51:46.109191 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1460/2899] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:51:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:51:46.113724 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2d6e3-fd96-4da7-9994-5458595954e3 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:46.124337 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d2d6e3-fd96-4da7-9994-5458595954e3 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:46.129529 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc79d5c-691e-4520-9426-a4bbb67bdaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:46.130136 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc79d5c-691e-4520-9426-a4bbb67bdaad admin admin] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:46.130445 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc79d5c-691e-4520-9426-a4bbb67bdaad admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:46.130714 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc79d5c-691e-4520-9426-a4bbb67bdaad admin admin] PATH_INFO: `/v3/projects` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:46.131333 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc79d5c-691e-4520-9426-a4bbb67bdaad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:46.133223 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc79d5c-691e-4520-9426-a4bbb67bdaad admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:46.161295 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1440/2900] 10.222.0.76 () {60 vars in 1153 bytes} [Wed Aug 24 15:51:46 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 15:51:46.165474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab91f5-4f2a-4bff-9c23-533e0b4e8c76 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:46.177351 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab91f5-4f2a-4bff-9c23-533e0b4e8c76 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:46.236788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454e4c0-5ffc-41e9-8b99-23023cb2cf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:46.237398 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454e4c0-5ffc-41e9-8b99-23023cb2cf34 admin admin] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:46.237696 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454e4c0-5ffc-41e9-8b99-23023cb2cf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:46.237965 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454e4c0-5ffc-41e9-8b99-23023cb2cf34 admin admin] PATH_INFO: `/v3/users` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:46.238627 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454e4c0-5ffc-41e9-8b99-23023cb2cf34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:46.240604 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454e4c0-5ffc-41e9-8b99-23023cb2cf34 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:46.263342 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1461/2901] 10.222.0.76 () {60 vars in 1144 bytes} [Wed Aug 24 15:51:46 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 15:51:46.267991 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc79d5c-691e-4520-9426-a4bbb67bdaad admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:46.278523 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc79d5c-691e-4520-9426-a4bbb67bdaad admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:46.283418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc7ab9-d649-4c71-9dfe-783ed4d43124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:46.284154 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc7ab9-d649-4c71-9dfe-783ed4d43124 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:46.284491 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc7ab9-d649-4c71-9dfe-783ed4d43124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:46.284831 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc7ab9-d649-4c71-9dfe-783ed4d43124 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:46.285575 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc7ab9-d649-4c71-9dfe-783ed4d43124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:46.287351 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dc7ab9-d649-4c71-9dfe-783ed4d43124 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:46.299779 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1441/2902] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:51:46.303423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454e4c0-5ffc-41e9-8b99-23023cb2cf34 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:46.314586 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454e4c0-5ffc-41e9-8b99-23023cb2cf34 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:46.319208 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7dbff-8448-40e2-babb-408dd01d4241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:46.320071 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7dbff-8448-40e2-babb-408dd01d4241 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:46.320267 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7dbff-8448-40e2-babb-408dd01d4241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:46.320500 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd7dbff-8448-40e2-babb-408dd01d4241 admin admin] PATH_INFO: `/v3/projects/2f315d7f756448208d5f37eea45336e3/users/5a4591de3c504d698215c7ecee33f27a/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:46.338410 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7dbff-8448-40e2-babb-408dd01d4241 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f315d7f756448208d5f37eea45336e3, user_id=5a4591de3c504d698215c7ecee33f27a, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:46.341101 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd7dbff-8448-40e2-babb-408dd01d4241 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:46.354903 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1462/2903] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:51:46 2022] PUT /identity/v3/projects/2f315d7f756448208d5f37eea45336e3/users/5a4591de3c504d698215c7ecee33f27a/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:51:46.359603 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc7ab9-d649-4c71-9dfe-783ed4d43124 admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:46.369997 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dc7ab9-d649-4c71-9dfe-783ed4d43124 admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:46.428124 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c22f169-34f3-4503-91dd-c85e883805f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:46.428732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22f169-34f3-4503-91dd-c85e883805f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:46.429009 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22f169-34f3-4503-91dd-c85e883805f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:46.429276 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c22f169-34f3-4503-91dd-c85e883805f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:46.429906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22f169-34f3-4503-91dd-c85e883805f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:46.431519 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c22f169-34f3-4503-91dd-c85e883805f0 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:46.442055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1442/2904] 10.222.0.76 () {58 vars in 1122 bytes} [Wed Aug 24 15:51:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 15:51:46.446304 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7dbff-8448-40e2-babb-408dd01d4241 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:46.456783 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd7dbff-8448-40e2-babb-408dd01d4241 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:46.461424 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb498295-8046-4dd5-8c12-7f171fa61e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:46.462047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb498295-8046-4dd5-8c12-7f171fa61e92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:46.462366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb498295-8046-4dd5-8c12-7f171fa61e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:46.462640 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb498295-8046-4dd5-8c12-7f171fa61e92 admin admin] PATH_INFO: `/v3/projects/2f315d7f756448208d5f37eea45336e3/users/5a4591de3c504d698215c7ecee33f27a/roles/b00780d2bbfb4c12b9bcab9312340187` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:46.464576 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb498295-8046-4dd5-8c12-7f171fa61e92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f315d7f756448208d5f37eea45336e3, user_id=5a4591de3c504d698215c7ecee33f27a, role_id=b00780d2bbfb4c12b9bcab9312340187)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:46.467333 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb498295-8046-4dd5-8c12-7f171fa61e92 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:46.477988 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1463/2905] 10.222.0.76 () {60 vars in 1483 bytes} [Wed Aug 24 15:51:46 2022] PUT /identity/v3/projects/2f315d7f756448208d5f37eea45336e3/users/5a4591de3c504d698215c7ecee33f27a/roles/b00780d2bbfb4c12b9bcab9312340187 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:51:51.200716 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8470e38-39c3-4a07-99be-c74df76e94e3 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:51.201265 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8470e38-39c3-4a07-99be-c74df76e94e3 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:51.201681 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8470e38-39c3-4a07-99be-c74df76e94e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:51.233538 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-b8470e38-39c3-4a07-99be-c74df76e94e3 None None] MFA Rules not processed for user `5a4591de3c504d698215c7ecee33f27a`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:51:51.237713 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-b8470e38-39c3-4a07-99be-c74df76e94e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:51.284010 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:51:51.316336 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1443/2906] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:51:51 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:51:51.333551 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb498295-8046-4dd5-8c12-7f171fa61e92 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:51:51.335477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cb498295-8046-4dd5-8c12-7f171fa61e92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:51:51.416970 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb498295-8046-4dd5-8c12-7f171fa61e92 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:51:51.422782 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc081eb-49cc-42fc-8bff-e44ae1dc06aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:51:51.423391 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc081eb-49cc-42fc-8bff-e44ae1dc06aa service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:51:51.423672 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc081eb-49cc-42fc-8bff-e44ae1dc06aa service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:51:51.423971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc081eb-49cc-42fc-8bff-e44ae1dc06aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:51:51.449334 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:51:51.482784 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc081eb-49cc-42fc-8bff-e44ae1dc06aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:51:51.484412 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc081eb-49cc-42fc-8bff-e44ae1dc06aa service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:51:51.493355 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1464/2907] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:51:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:52:01.314368 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=12 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:52:01.489269 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=16 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:52:10.897910 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fb5bb-e617-4bda-8980-28e6b729aa38 None None] REQUEST_METHOD: `POST` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:52:10.898459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fb5bb-e617-4bda-8980-28e6b729aa38 None None] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:52:10.898459 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470fb5bb-e617-4bda-8980-28e6b729aa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:52:10.917375 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.auth.core [None req-470fb5bb-e617-4bda-8980-28e6b729aa38 None None] MFA Rules not processed for user `1151eb3a305e4c18b8cc5717379bce63`. Rule list: `[]` (Enabled: `True`). {{(pid=81620) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:52:10.918605 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-470fb5bb-e617-4bda-8980-28e6b729aa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:52:10.925649 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1444/2908] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:52:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:52:11.680455 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc081eb-49cc-42fc-8bff-e44ae1dc06aa service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:52:11.681883 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-fcc081eb-49cc-42fc-8bff-e44ae1dc06aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:52:11.752150 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc081eb-49cc-42fc-8bff-e44ae1dc06aa service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:52:11.756988 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef62f6-1b9e-415d-ae83-bfa05223e794 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:52:11.757584 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef62f6-1b9e-415d-ae83-bfa05223e794 service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:52:11.757884 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef62f6-1b9e-415d-ae83-bfa05223e794 service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:52:11.758152 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbef62f6-1b9e-415d-ae83-bfa05223e794 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:52:11.759861 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-cbef62f6-1b9e-415d-ae83-bfa05223e794 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:52:11.771600 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef62f6-1b9e-415d-ae83-bfa05223e794 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:52:11.773156 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbef62f6-1b9e-415d-ae83-bfa05223e794 service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:52:11.783503 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1465/2909] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:52:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:52:14.950416 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fb5bb-e617-4bda-8980-28e6b729aa38 None None] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:52:15.009785 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470fb5bb-e617-4bda-8980-28e6b729aa38 None None] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:52:15.013321 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efa0f3-73de-400b-ae32-d65cd59fe5ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:52:15.013924 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efa0f3-73de-400b-ae32-d65cd59fe5ec service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:52:15.014270 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efa0f3-73de-400b-ae32-d65cd59fe5ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:52:15.014542 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17efa0f3-73de-400b-ae32-d65cd59fe5ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:52:15.016313 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-17efa0f3-73de-400b-ae32-d65cd59fe5ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:52:15.030679 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efa0f3-73de-400b-ae32-d65cd59fe5ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:52:15.032145 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17efa0f3-73de-400b-ae32-d65cd59fe5ec service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:52:15.044438 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1445/2910] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:52:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:52:21.778738 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=32 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:52:25.040605 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=29 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:52:26.036132 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef62f6-1b9e-415d-ae83-bfa05223e794 service placement] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:52:26.050819 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbef62f6-1b9e-415d-ae83-bfa05223e794 service placement] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:52:26.056263 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8359627-0e44-4dcd-b040-5e99b36f364e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:52:26.056862 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8359627-0e44-4dcd-b040-5e99b36f364e service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:52:26.057138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8359627-0e44-4dcd-b040-5e99b36f364e service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:52:26.057406 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8359627-0e44-4dcd-b040-5e99b36f364e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:52:26.059117 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-c8359627-0e44-4dcd-b040-5e99b36f364e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:52:26.072029 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8359627-0e44-4dcd-b040-5e99b36f364e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:52:26.073601 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8359627-0e44-4dcd-b040-5e99b36f364e service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:52:26.084444 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1466/2911] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:52:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:52:29.361580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efa0f3-73de-400b-ae32-d65cd59fe5ec service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:52:29.363066 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-17efa0f3-73de-400b-ae32-d65cd59fe5ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:52:29.373617 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17efa0f3-73de-400b-ae32-d65cd59fe5ec service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:52:29.377685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164fe1d-97af-4923-a4f9-1a92df04d78a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:52:29.378581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164fe1d-97af-4923-a4f9-1a92df04d78a service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:52:29.378977 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164fe1d-97af-4923-a4f9-1a92df04d78a service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:52:29.379378 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c164fe1d-97af-4923-a4f9-1a92df04d78a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:52:29.388943 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164fe1d-97af-4923-a4f9-1a92df04d78a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:52:29.390387 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c164fe1d-97af-4923-a4f9-1a92df04d78a service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:52:29.399628 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1446/2912] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:52:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:52:36.080047 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:52:39.395504 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:53:55.291512 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8359627-0e44-4dcd-b040-5e99b36f364e service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:53:55.301346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8359627-0e44-4dcd-b040-5e99b36f364e service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:53:55.304859 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b7ffd8-3471-4d4e-9117-1fb4946100d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:53:55.305463 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7ffd8-3471-4d4e-9117-1fb4946100d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:53:55.306331 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7ffd8-3471-4d4e-9117-1fb4946100d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:53:55.306687 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b7ffd8-3471-4d4e-9117-1fb4946100d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:53:55.309199 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-d8b7ffd8-3471-4d4e-9117-1fb4946100d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:53:55.318662 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7ffd8-3471-4d4e-9117-1fb4946100d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:53:55.320069 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b7ffd8-3471-4d4e-9117-1fb4946100d3 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:53:55.328171 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1467/2913] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:53:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:54:05.324602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:54:47.401110 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164fe1d-97af-4923-a4f9-1a92df04d78a service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:54:47.416407 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c164fe1d-97af-4923-a4f9-1a92df04d78a service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:54:47.424701 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a088f-ae78-4e70-927a-f7e875e55491 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:54:47.425366 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a088f-ae78-4e70-927a-f7e875e55491 service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:54:47.425647 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a088f-ae78-4e70-927a-f7e875e55491 service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:54:47.425917 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a088f-ae78-4e70-927a-f7e875e55491 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:54:47.427735 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-da9a088f-ae78-4e70-927a-f7e875e55491 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:54:47.448285 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a088f-ae78-4e70-927a-f7e875e55491 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:54:47.450121 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9a088f-ae78-4e70-927a-f7e875e55491 service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:54:47.464590 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1447/2914] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:54:47 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:54:48.312722 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a68df9-1597-4157-9156-d1c496541a71 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:54:48.313250 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a68df9-1597-4157-9156-d1c496541a71 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:54:48.313250 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a68df9-1597-4157-9156-d1c496541a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:54:48.332620 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-18a68df9-1597-4157-9156-d1c496541a71 None None] MFA Rules not processed for user `1151eb3a305e4c18b8cc5717379bce63`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:54:48.333814 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-18a68df9-1597-4157-9156-d1c496541a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:54:48.341477 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1468/2915] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:54:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:54:48.658395 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a088f-ae78-4e70-927a-f7e875e55491 service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:54:48.667719 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a088f-ae78-4e70-927a-f7e875e55491 service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:54:48.671277 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03897d59-a5eb-43c7-a85f-fd0b95f50b61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:54:48.672127 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03897d59-a5eb-43c7-a85f-fd0b95f50b61 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:54:48.672404 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03897d59-a5eb-43c7-a85f-fd0b95f50b61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:54:48.672672 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03897d59-a5eb-43c7-a85f-fd0b95f50b61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:54:48.674412 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-03897d59-a5eb-43c7-a85f-fd0b95f50b61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:54:48.683745 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03897d59-a5eb-43c7-a85f-fd0b95f50b61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:54:48.685141 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03897d59-a5eb-43c7-a85f-fd0b95f50b61 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:54:48.693055 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1448/2916] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:54:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:54:58.327374 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=3 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:54:58.689477 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=12 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:55:01.562441 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a68df9-1597-4157-9156-d1c496541a71 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:55:01.564020 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-18a68df9-1597-4157-9156-d1c496541a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:55:01.622082 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a68df9-1597-4157-9156-d1c496541a71 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:55:01.627727 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e873978-6e08-4955-b0d1-a7146f7e20b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:55:01.628325 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e873978-6e08-4955-b0d1-a7146f7e20b2 service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:55:01.628602 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e873978-6e08-4955-b0d1-a7146f7e20b2 service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:55:01.628872 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e873978-6e08-4955-b0d1-a7146f7e20b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:55:01.630595 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-9e873978-6e08-4955-b0d1-a7146f7e20b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:55:01.642748 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e873978-6e08-4955-b0d1-a7146f7e20b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:55:01.644346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e873978-6e08-4955-b0d1-a7146f7e20b2 service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:55:01.654286 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1469/2917] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:55:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:55:11.650034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=24 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:57:00.529207 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03897d59-a5eb-43c7-a85f-fd0b95f50b61 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:57:00.531015 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-03897d59-a5eb-43c7-a85f-fd0b95f50b61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:57:00.613210 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03897d59-a5eb-43c7-a85f-fd0b95f50b61 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:57:00.617355 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe9381-7a84-4dc1-81b7-aed5ee4ded4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '194c7492cf954a6a8dd21de66aa24fec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:57:00.617945 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe9381-7a84-4dc1-81b7-aed5ee4ded4b service cinder] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:57:00.618236 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe9381-7a84-4dc1-81b7-aed5ee4ded4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:57:00.618513 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfe9381-7a84-4dc1-81b7-aed5ee4ded4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:57:00.626728 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe9381-7a84-4dc1-81b7-aed5ee4ded4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:57:00.628148 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfe9381-7a84-4dc1-81b7-aed5ee4ded4b service cinder] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:57:00.636685 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1449/2918] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:57:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:57:10.632822 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=39 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:57:11.633412 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e5831-fa76-4318-9c5c-5d9150a0d833 None None] REQUEST_METHOD: `POST` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:57:11.633971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e5831-fa76-4318-9c5c-5d9150a0d833 None None] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:57:11.633971 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651e5831-fa76-4318-9c5c-5d9150a0d833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:57:11.652753 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.auth.core [None req-651e5831-fa76-4318-9c5c-5d9150a0d833 None None] MFA Rules not processed for user `1151eb3a305e4c18b8cc5717379bce63`. Rule list: `[]` (Enabled: `True`). {{(pid=81619) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 15:57:11.653942 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-651e5831-fa76-4318-9c5c-5d9150a0d833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:57:11.660990 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1470/2919] 10.222.0.76 () {58 vars in 958 bytes} [Wed Aug 24 15:57:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 15:57:12.415608 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe9381-7a84-4dc1-81b7-aed5ee4ded4b service cinder] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:57:12.428301 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfe9381-7a84-4dc1-81b7-aed5ee4ded4b service cinder] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:57:12.433241 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee8e7c-b303-4aea-bde8-0457f6c5acb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:57:12.433842 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee8e7c-b303-4aea-bde8-0457f6c5acb3 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:57:12.434117 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee8e7c-b303-4aea-bde8-0457f6c5acb3 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:57:12.434418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee8e7c-b303-4aea-bde8-0457f6c5acb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:57:12.443597 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee8e7c-b303-4aea-bde8-0457f6c5acb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:57:12.444964 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee8e7c-b303-4aea-bde8-0457f6c5acb3 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:57:12.455550 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1450/2920] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:57:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:57:14.632619 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e5831-fa76-4318-9c5c-5d9150a0d833 None None] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:57:14.658655 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651e5831-fa76-4318-9c5c-5d9150a0d833 None None] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:57:14.662231 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8931c85a-832d-44a9-b93e-4b5c6e52fecb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:57:14.663010 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931c85a-832d-44a9-b93e-4b5c6e52fecb service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:57:14.663146 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931c85a-832d-44a9-b93e-4b5c6e52fecb service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:57:14.663380 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8931c85a-832d-44a9-b93e-4b5c6e52fecb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:57:14.675445 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8931c85a-832d-44a9-b93e-4b5c6e52fecb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:57:14.676875 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8931c85a-832d-44a9-b93e-4b5c6e52fecb service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:57:14.688365 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1471/2921] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:57:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 15:57:22.450875 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:57:24.683341 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=13 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:57:27.333660 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee8e7c-b303-4aea-bde8-0457f6c5acb3 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:57:27.367034 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee8e7c-b303-4aea-bde8-0457f6c5acb3 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:57:27.372506 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd47c32-32d3-4532-9583-23e8597e1f9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:57:27.373104 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd47c32-32d3-4532-9583-23e8597e1f9f service nova] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:57:27.373376 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd47c32-32d3-4532-9583-23e8597e1f9f service nova] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:57:27.373643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd47c32-32d3-4532-9583-23e8597e1f9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:57:27.382855 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd47c32-32d3-4532-9583-23e8597e1f9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:57:27.384418 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd47c32-32d3-4532-9583-23e8597e1f9f service nova] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:57:27.397004 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1451/2922] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:57:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:57:30.434077 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8931c85a-832d-44a9-b93e-4b5c6e52fecb service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:57:30.442893 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8931c85a-832d-44a9-b93e-4b5c6e52fecb service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:57:30.446474 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a46ee-fc46-4b28-88d3-8186cfb71824 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:57:30.447066 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a46ee-fc46-4b28-88d3-8186cfb71824 service neutron] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:57:30.447342 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a46ee-fc46-4b28-88d3-8186cfb71824 service neutron] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:57:30.447610 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452a46ee-fc46-4b28-88d3-8186cfb71824 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:57:30.456408 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a46ee-fc46-4b28-88d3-8186cfb71824 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:57:30.457833 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452a46ee-fc46-4b28-88d3-8186cfb71824 service neutron] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:57:30.467172 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1472/2923] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:57:30 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:57:37.390998 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=13 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:57:40.463034 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=6 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:59:02.288307 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd47c32-32d3-4532-9583-23e8597e1f9f service nova] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:59:02.298821 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd47c32-32d3-4532-9583-23e8597e1f9f service nova] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:59:02.302411 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57aa0d-d30e-4135-a7b5-9c928e5c1180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:59:02.303003 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57aa0d-d30e-4135-a7b5-9c928e5c1180 service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:59:02.303279 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57aa0d-d30e-4135-a7b5-9c928e5c1180 service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:59:02.303548 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57aa0d-d30e-4135-a7b5-9c928e5c1180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:59:02.310732 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57aa0d-d30e-4135-a7b5-9c928e5c1180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:59:02.312138 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57aa0d-d30e-4135-a7b5-9c928e5c1180 service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:59:02.320047 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1452/2924] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:59:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:59:12.316568 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=6 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:59:22.033860 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a46ee-fc46-4b28-88d3-8186cfb71824 service neutron] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:59:22.046368 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452a46ee-fc46-4b28-88d3-8186cfb71824 service neutron] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:59:22.052452 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49162ccf-6ad8-455d-b207-8c27eb73e7cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8a5fd31a2db944f596912e4f0aa26835', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'member', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:59:22.053037 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49162ccf-6ad8-455d-b207-8c27eb73e7cf service nova] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:59:22.053309 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49162ccf-6ad8-455d-b207-8c27eb73e7cf service nova] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:59:22.053573 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49162ccf-6ad8-455d-b207-8c27eb73e7cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:59:22.055366 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-49162ccf-6ad8-455d-b207-8c27eb73e7cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:59:22.084131 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49162ccf-6ad8-455d-b207-8c27eb73e7cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:59:22.085729 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49162ccf-6ad8-455d-b207-8c27eb73e7cf service nova] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:59:22.096890 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1473/2925] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:59:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:59:32.092259 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=11 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:59:34.212818 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57aa0d-d30e-4135-a7b5-9c928e5c1180 service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:59:34.221537 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57aa0d-d30e-4135-a7b5-9c928e5c1180 service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:59:34.225171 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899bf5c-c993-4441-89b8-78b5486ddb2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5427bd0ac88042338b956c10cfd07baa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:59:34.225844 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899bf5c-c993-4441-89b8-78b5486ddb2f service neutron] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:59:34.226046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899bf5c-c993-4441-89b8-78b5486ddb2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:59:34.226357 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899bf5c-c993-4441-89b8-78b5486ddb2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:59:34.287626 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1899bf5c-c993-4441-89b8-78b5486ddb2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:59:34.289117 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1899bf5c-c993-4441-89b8-78b5486ddb2f service neutron] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:59:34.299004 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1453/2926] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:59:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:59:37.414934 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49162ccf-6ad8-455d-b207-8c27eb73e7cf service nova] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:59:37.436147 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49162ccf-6ad8-455d-b207-8c27eb73e7cf service nova] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:59:37.450603 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3cfbe-16b1-4a6f-8864-484689cfb571 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:59:37.451273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3cfbe-16b1-4a6f-8864-484689cfb571 service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:59:37.451498 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3cfbe-16b1-4a6f-8864-484689cfb571 service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:59:37.451770 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3cfbe-16b1-4a6f-8864-484689cfb571 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:59:37.453587 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-a3a3cfbe-16b1-4a6f-8864-484689cfb571 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:59:37.467296 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3cfbe-16b1-4a6f-8864-484689cfb571 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:59:37.468916 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3cfbe-16b1-4a6f-8864-484689cfb571 service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:59:37.482499 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1474/2927] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:59:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:59:40.543250 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899bf5c-c993-4441-89b8-78b5486ddb2f service neutron] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:59:40.559799 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899bf5c-c993-4441-89b8-78b5486ddb2f service neutron] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:59:40.565380 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412d02b-51f2-473a-8bd2-b208bc602154 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:59:40.566076 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412d02b-51f2-473a-8bd2-b208bc602154 service placement] REQUEST_METHOD: `GET` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:59:40.566498 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412d02b-51f2-473a-8bd2-b208bc602154 service placement] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:59:40.566841 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7412d02b-51f2-473a-8bd2-b208bc602154 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:59:40.568693 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-7412d02b-51f2-473a-8bd2-b208bc602154 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:59:40.582063 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412d02b-51f2-473a-8bd2-b208bc602154 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:59:40.583793 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7412d02b-51f2-473a-8bd2-b208bc602154 service placement] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:59:40.602697 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1454/2928] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 15:59:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 15:59:40.603686 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3cfbe-16b1-4a6f-8864-484689cfb571 service placement] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:59:40.616423 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3cfbe-16b1-4a6f-8864-484689cfb571 service placement] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:59:40.621296 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108f1d6-aaa7-425a-8ccd-87684faf676a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:59:40.622289 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108f1d6-aaa7-425a-8ccd-87684faf676a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:59:40.622668 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108f1d6-aaa7-425a-8ccd-87684faf676a admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:59:40.623139 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2108f1d6-aaa7-425a-8ccd-87684faf676a admin admin] PATH_INFO: `/v3/users/1151eb3a305e4c18b8cc5717379bce63` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:59:40.624731 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108f1d6-aaa7-425a-8ccd-87684faf676a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1151eb3a305e4c18b8cc5717379bce63)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:59:40.626702 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2108f1d6-aaa7-425a-8ccd-87684faf676a admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:59:40.675202 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2108f1d6-aaa7-425a-8ccd-87684faf676a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1151eb3a305e4c18b8cc5717379bce63'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:40.682338 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2108f1d6-aaa7-425a-8ccd-87684faf676a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1151eb3a305e4c18b8cc5717379bce63'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:40.694964 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2108f1d6-aaa7-425a-8ccd-87684faf676a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1151eb3a305e4c18b8cc5717379bce63'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:40.695604 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2108f1d6-aaa7-425a-8ccd-87684faf676a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1151eb3a305e4c18b8cc5717379bce63'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:40.702683 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-2108f1d6-aaa7-425a-8ccd-87684faf676a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1151eb3a305e4c18b8cc5717379bce63'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:40.709490 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1475/2929] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:59:40 2022] DELETE /identity/v3/users/1151eb3a305e4c18b8cc5717379bce63 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:59:40.887616 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412d02b-51f2-473a-8bd2-b208bc602154 service placement] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:59:40.898906 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7412d02b-51f2-473a-8bd2-b208bc602154 service placement] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:59:40.961195 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ef235-da54-4a63-a1fc-a6eab3a23a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:59:40.961825 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ef235-da54-4a63-a1fc-a6eab3a23a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:59:40.962103 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ef235-da54-4a63-a1fc-a6eab3a23a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:59:40.962564 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ef235-da54-4a63-a1fc-a6eab3a23a6b admin admin] PATH_INFO: `/v3/projects/ae3022891c914ea18ffacc414b55fb7e` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:59:40.963695 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ef235-da54-4a63-a1fc-a6eab3a23a6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae3022891c914ea18ffacc414b55fb7e)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:59:40.965571 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ef235-da54-4a63-a1fc-a6eab3a23a6b admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:59:41.013193 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-315ef235-da54-4a63-a1fc-a6eab3a23a6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae3022891c914ea18ffacc414b55fb7e', 'request_id': 'req-315ef235-da54-4a63-a1fc-a6eab3a23a6b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:41.018836 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-315ef235-da54-4a63-a1fc-a6eab3a23a6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae3022891c914ea18ffacc414b55fb7e', 'request_id': 'req-315ef235-da54-4a63-a1fc-a6eab3a23a6b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:41.026580 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-315ef235-da54-4a63-a1fc-a6eab3a23a6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae3022891c914ea18ffacc414b55fb7e', 'request_id': 'req-315ef235-da54-4a63-a1fc-a6eab3a23a6b'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:41.038643 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-315ef235-da54-4a63-a1fc-a6eab3a23a6b admin admin] The token cache is being invalidate because project ae3022891c914ea18ffacc414b55fb7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:59:41.038929 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-315ef235-da54-4a63-a1fc-a6eab3a23a6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:41.039561 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-315ef235-da54-4a63-a1fc-a6eab3a23a6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:41.041043 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1455/2930] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:59:40 2022] DELETE /identity/v3/projects/ae3022891c914ea18ffacc414b55fb7e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:59:41.046101 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108f1d6-aaa7-425a-8ccd-87684faf676a admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:59:41.047583 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-2108f1d6-aaa7-425a-8ccd-87684faf676a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:59:41.062487 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2108f1d6-aaa7-425a-8ccd-87684faf676a admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:59:41.122346 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d8f14-a183-4abc-924d-6ef74ad15726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:59:41.122972 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d8f14-a183-4abc-924d-6ef74ad15726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:59:41.123246 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d8f14-a183-4abc-924d-6ef74ad15726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:59:41.123515 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280d8f14-a183-4abc-924d-6ef74ad15726 admin admin] PATH_INFO: `/v3/users/5a4591de3c504d698215c7ecee33f27a` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:59:41.124628 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d8f14-a183-4abc-924d-6ef74ad15726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a4591de3c504d698215c7ecee33f27a)` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:59:41.126561 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280d8f14-a183-4abc-924d-6ef74ad15726 admin admin] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:59:41.172905 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-280d8f14-a183-4abc-924d-6ef74ad15726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a4591de3c504d698215c7ecee33f27a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:41.181138 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-280d8f14-a183-4abc-924d-6ef74ad15726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a4591de3c504d698215c7ecee33f27a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:41.186347 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:59:41.195273 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-280d8f14-a183-4abc-924d-6ef74ad15726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a4591de3c504d698215c7ecee33f27a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:41.196101 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-280d8f14-a183-4abc-924d-6ef74ad15726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a4591de3c504d698215c7ecee33f27a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:41.203050 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.notifications [None req-280d8f14-a183-4abc-924d-6ef74ad15726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a4591de3c504d698215c7ecee33f27a'} {{(pid=81619) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:41.208913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1476/2931] 10.222.0.76 () {58 vars in 1224 bytes} [Wed Aug 24 15:59:41 2022] DELETE /identity/v3/users/5a4591de3c504d698215c7ecee33f27a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 15:59:41.379063 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ef235-da54-4a63-a1fc-a6eab3a23a6b admin admin] Authenticating user token {{(pid=81620) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 15:59:41.380581 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.fernet_utils [None req-315ef235-da54-4a63-a1fc-a6eab3a23a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81620) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 15:59:41.395647 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ef235-da54-4a63-a1fc-a6eab3a23a6b admin admin] Validating token access rules against request {{(pid=81620) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 15:59:41.405046 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:59:41.455514 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc34bfb-c92f-4208-b650-a2504fda21f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '793b87d316d544f9aecda76f798927dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'e70c51040e434074a3d61be0b7c8533b', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81620) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 15:59:41.456138 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc34bfb-c92f-4208-b650-a2504fda21f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 15:59:41.456413 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc34bfb-c92f-4208-b650-a2504fda21f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 15:59:41.456680 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc34bfb-c92f-4208-b650-a2504fda21f9 admin admin] PATH_INFO: `/v3/projects/2f315d7f756448208d5f37eea45336e3` {{(pid=81620) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 15:59:41.457776 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc34bfb-c92f-4208-b650-a2504fda21f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f315d7f756448208d5f37eea45336e3)` {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 15:59:41.459908 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc34bfb-c92f-4208-b650-a2504fda21f9 admin admin] RBAC: Authorization granted {{(pid=81620) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 15:59:41.505192 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ecc34bfb-c92f-4208-b650-a2504fda21f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f315d7f756448208d5f37eea45336e3', 'request_id': 'req-ecc34bfb-c92f-4208-b650-a2504fda21f9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:41.510900 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ecc34bfb-c92f-4208-b650-a2504fda21f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f315d7f756448208d5f37eea45336e3', 'request_id': 'req-ecc34bfb-c92f-4208-b650-a2504fda21f9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:41.518324 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ecc34bfb-c92f-4208-b650-a2504fda21f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f315d7f756448208d5f37eea45336e3', 'request_id': 'req-ecc34bfb-c92f-4208-b650-a2504fda21f9'} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:41.529280 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ecc34bfb-c92f-4208-b650-a2504fda21f9 admin admin] The token cache is being invalidate because project 2f315d7f756448208d5f37eea45336e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81620) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 15:59:41.529565 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ecc34bfb-c92f-4208-b650-a2504fda21f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:41.530194 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG keystone.notifications [None req-ecc34bfb-c92f-4208-b650-a2504fda21f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81620) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 15:59:41.531689 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: [pid: 81620|app: 0|req: 1456/2932] 10.222.0.76 () {58 vars in 1233 bytes} [Wed Aug 24 15:59:41 2022] DELETE /identity/v3/projects/2f315d7f756448208d5f37eea45336e3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 15:59:51.205788 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=9,keystone:DELETE=1 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 15:59:51.526419 dv-cinder-848900-6-iscsi devstack@keystone.service[81620]: DEBUG dbcounter [-] [81620] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81620) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 16:00:08.565024 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d8f14-a183-4abc-924d-6ef74ad15726 admin admin] Authenticating user token {{(pid=81619) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 16:00:08.566469 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-280d8f14-a183-4abc-924d-6ef74ad15726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:00:08.634825 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280d8f14-a183-4abc-924d-6ef74ad15726 admin admin] Validating token access rules against request {{(pid=81619) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 16:00:08.692814 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63568f98-3144-4ebd-aa41-674d7b00824a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb745b98e3ae4723bad44428646f83e2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2c010096b77f4cd3b2f32086b1ba7ea7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81619) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 16:00:08.693416 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63568f98-3144-4ebd-aa41-674d7b00824a service placement] REQUEST_METHOD: `GET` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 16:00:08.693694 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63568f98-3144-4ebd-aa41-674d7b00824a service placement] SCRIPT_NAME: `/identity` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 16:00:08.693966 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63568f98-3144-4ebd-aa41-674d7b00824a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81619) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 16:00:08.695744 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.fernet_utils [None req-63568f98-3144-4ebd-aa41-674d7b00824a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81619) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 16:00:08.706913 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63568f98-3144-4ebd-aa41-674d7b00824a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 16:00:08.708480 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63568f98-3144-4ebd-aa41-674d7b00824a service placement] RBAC: Authorization granted {{(pid=81619) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 16:00:08.718319 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: [pid: 81619|app: 0|req: 1477/2933] 10.222.0.76 () {60 vars in 1365 bytes} [Wed Aug 24 16:00:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 16:00:18.714045 dv-cinder-848900-6-iscsi devstack@keystone.service[81619]: DEBUG dbcounter [-] [81619] Writing DB stats keystone:SELECT=58 {{(pid=81619) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}